/LMAR=0/XLINE=3/FONT#0=BAXL30/FONT#1=BAXI30/FONT#2=BASB30/FONT#3=SUB/FONT#4=SUP/FONT#5=BASL35/FONT#6=NGR25/FONT#7=XMAS25



␈↓ α∧␈↓α␈↓ ¬8AN UNREASONABLE BOOK



␈↓ α∧␈↓Joseph␈α∂Weizenbaum,␈α∞␈↓↓Computer␈α∂Power␈α∞and␈α∂Human␈α∞Reason␈↓,␈α∂W.␈α∞H.␈α∂Freeman␈α∞Co.,␈α∂San␈α∞Francisco
␈↓ α∧␈↓1975

␈↓ α∧␈↓␈↓ αTThis␈αmoralistic␈αand␈α
incoherent␈αbook␈αuses␈α
computer␈αscience␈αand␈α
technology␈αas␈αan␈α
illustration
␈↓ α∧␈↓to␈α⊂support␈α⊂the␈α⊂view␈α⊂promoted␈α⊂by␈α⊂Lewis␈α⊂Mumford,␈α⊂Theodore␈α⊂Roszak,␈α⊂and␈α⊂Jacques␈α⊃Ellul,␈α⊂that
␈↓ α∧␈↓science␈αhas␈αled␈αto␈αan␈α"immoral"␈αview␈αof␈αman␈αand␈αthe␈αworld.␈α It␈αalso␈αowes␈αmuch␈αin␈αrhetorical␈αstyle
␈↓ α∧␈↓and␈α∩political␈α∩presupposition␈α∩to␈α∩the␈α∩new␈α∩left.␈α∩ I␈α∩am␈α∩frightened␈α∩by␈α∩its␈α∩arguments␈α∪that␈α∩certain
␈↓ α∧␈↓research␈αshould␈αnot␈αbe␈αdone␈αif␈αit␈αis␈αbased␈αon␈αor␈αmight␈αresult␈αin␈αan␈α"obscene"␈αpicture␈αof␈αthe␈αworld
␈↓ α∧␈↓and␈αman.␈α Worse␈αyet,␈αhis␈αnotion␈αof␈α"obscenity"␈αis␈αvague␈αenough␈αto␈αadmit␈αarbitrary␈αinterpretations
␈↓ α∧␈↓by activist bureaucrats.

␈↓ α∧␈↓α␈↓ β>IT'S HARD TO FIGURE OUT WHAT HE REALLY BELIEVES ...

␈↓ α∧␈↓␈↓ αTWeizenbaum's␈α∀style␈α∀involves␈α∀making␈α∃extreme␈α∀statements␈α∀which␈α∀are␈α∀later␈α∃qualified␈α∀by
␈↓ α∧␈↓contradictory␈α∞statements.␈α∂ Therefore,␈α∞almost␈α∂any␈α∞quotation␈α∂is␈α∞out␈α∞of␈α∂context,␈α∞so␈α∂it␈α∞is␈α∂difficult␈α∞to
␈↓ α∧␈↓summarize his contentions accurately.

␈↓ α∧␈↓␈↓ αTThe following passages illustrate the difficulty:

␈↓ α∧␈↓␈↓ αT␈↓↓"In␈α∩1935,␈α∩Michael␈α∪Polanyi"␈↓,␈α∩[British␈α∩chemist␈α∩and␈α∪philosopher␈α∩of␈α∩science,␈α∩was␈α∪told␈α∩by]
␈↓ α∧␈↓␈↓↓"Nicolai␈α⊂Bukharin,␈α∂one␈α⊂of␈α∂the␈α⊂leading␈α∂theoreticians␈α⊂of␈α∂the␈α⊂Russian␈α∂Communist␈α⊂party,␈α⊂...␈α∂ [that]
␈↓ α∧␈↓↓'under␈α
socialism␈α
the␈α
conception␈α
of␈α
science␈α
pursued␈α
for␈α
its␈α
own␈α
sake␈α
would␈α
disappear,␈α
for␈αthe␈α
interests
␈↓ α∧␈↓↓of␈α⊂scientists␈α⊂would␈α⊂spontaneously␈α⊂turn␈α⊂to␈α⊃the␈α⊂problems␈α⊂of␈α⊂the␈α⊂current␈α⊂Five␈α⊂Year␈α⊃Plan.'␈α⊂Polanyi
␈↓ α∧␈↓↓sensed␈α
then␈α
that␈α'the␈α
scientific␈α
outlook␈α
appeared␈αto␈α
have␈α
produced␈αa␈α
mechanical␈α
conception␈α
of␈αman
␈↓ α∧␈↓↓and␈αhistory␈αin␈αwhich␈αthere␈αwas␈αno␈αplace␈αfor␈αscience␈αitself.'␈αAnd␈αfurther␈αthat␈α'this␈αconception␈αdenied
␈↓ α∧␈↓↓altogether␈α∂any␈α∂intrinsic␈α⊂power␈α∂to␈α∂thought␈α⊂and␈α∂thus␈α∂denied␈α∂any␈α⊂grounds␈α∂for␈α∂claiming␈α⊂freedom␈α∂of
␈↓ α∧␈↓↓thought.'"␈↓␈α-␈αfrom␈αpage␈α1.␈α Well,␈αthat's␈αclear␈αenough;␈αWeizenbaum␈αfavors␈αfreedom␈αof␈αthought␈αand
␈↓ α∧␈↓science and is worried about threats to them.  But on page 265, we have

␈↓ α∧␈↓␈↓ αT␈↓↓"Scientists␈α
who␈α∞continue␈α
to␈α
prattle␈α∞on␈α
about␈α
'knowledge␈α∞for␈α
its␈α
own␈α∞sake'␈α
in␈α
order␈α∞to␈α
exploit
␈↓ α∧␈↓↓that␈αslogan␈α
for␈αtheir␈α
self-serving␈αends␈α
have␈αdetached␈αscience␈α
and␈αknowledge␈α
from␈αany␈α
contact␈αwith
␈↓ α∧␈↓↓the␈αreal␈αworld".␈↓␈αHere␈αWeizenbaum␈αseems␈αto␈αbe␈αagainst␈αpure␈αscience,␈αi.e.␈α research␈αmotivated␈αsolely
␈↓ α∧␈↓by curiosity.  We also have

␈↓ α∧␈↓␈↓ αT␈↓↓"With␈α∀few␈α∃exceptions,␈α∀there␈α∀have␈α∃been␈α∀no␈α∃results,␈α∀from␈α∀over␈α∃twenty␈α∀years␈α∃of␈α∀artificial
␈↓ α∧␈↓↓intelligence␈α
research,␈α
that␈α
have␈α
found␈α
their␈α
way␈α
into␈α
industry␈α
generally␈α
or␈α
into␈α
the␈αcomputer␈α
industry
␈↓ α∧␈↓↓in␈α∂particular"␈↓.␈α∞ -␈α∂page␈α∞229␈α∂This␈α∂again␈α∞suggests␈α∂that␈α∞industrial␈α∂results␈α∞are␈α∂necessary␈α∂to␈α∞validate
␈↓ α∧␈↓science.

␈↓ α∧␈↓␈↓ αT␈↓↓"Science␈αpromised␈αman␈αpower.␈α But␈αas␈αso␈αoften␈αhappens␈αwhen␈αpeople␈αare␈αseduced␈αby␈αpromises
␈↓ α∧␈↓↓of power ... the price actually paid is servitude and impotence"␈↓.

␈↓ α∧␈↓␈↓ αT␈↓↓"I␈αwill,␈αin␈αwhat␈αfollows,␈αtry␈αto␈αmaintain␈αthe␈αposition␈αthat␈αthere␈αis␈αnothing␈αwrong␈αwith␈αviewing


␈↓ α∧␈↓␈↓ ε|1␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓↓man␈α∞as␈α∞an␈α∞information␈α
processor␈α∞(or␈α∞indeed␈α∞as␈α∞anything␈α
else)␈α∞nor␈α∞with␈α∞attempting␈α∞to␈α
understand
␈↓ α∧␈↓↓him␈α
from␈α
that␈α
perspective,␈α∞providing,␈α
however,␈α
that␈α
we␈α
never␈α∞act␈α
as␈α
though␈α
any␈α∞single␈α
perspective
␈↓ α∧␈↓↓can comprehend the whole man."␈↓ - page 140.  We can certainly live with that, but

␈↓ α∧␈↓␈↓ αT␈↓↓"Not␈αonly␈αhas␈αour␈αunbounded␈αfeeding␈αon␈αscience␈αcaused␈αus␈αto␈αbecome␈αdependent␈αon␈αit,␈αbut,␈αas
␈↓ α∧␈↓↓happens␈α
with␈α
many␈α
other␈α
drugs␈α
taken␈α
in␈α
increasing␈α
dosages,␈α
science␈α
has␈α
been␈α∞gradually␈α
converted
␈↓ α∧␈↓↓into a slow acting poison".␈↓ - page 13.  These are qualified by

␈↓ α∧␈↓␈↓ αT␈↓↓"I␈α
argue␈αfor␈α
the␈αrational␈α
use␈αof␈α
science␈αand␈α
technology,␈αnot␈α
for␈αits␈α
mystification,␈αlet␈α
alone␈αits
␈↓ α∧␈↓↓abandonment"␈↓. - page 256

␈↓ α∧␈↓␈↓ αTIn␈α⊂reference␈α∂to␈α⊂the␈α∂proposal␈α⊂for␈α∂a␈α⊂moratorium␈α∂on␈α⊂certain␈α∂experiments␈α⊂with␈α∂recombinant
␈↓ α∧␈↓DNA␈αbecause␈αthey␈αmight␈αbe␈αdangerous,␈αwe␈αhave␈α␈↓↓"Theirs␈αis␈αcertainly␈αa␈αstep␈αin␈αthe␈αright␈αdirection,
␈↓ α∧␈↓↓and␈α∞their␈α∞initiative␈α∞is␈α∂to␈α∞be␈α∞applauded.␈α∞ Still,␈α∞one␈α∂may␈α∞ask,␈α∞why␈α∞do␈α∞they␈α∂feel␈α∞they␈α∞have␈α∞to␈α∂give␈α∞a
␈↓ α∧␈↓↓reason␈α
for␈α
what␈α
they␈αrecommend␈α
at␈α
all?␈α
 Is␈αnot␈α
the␈α
overriding␈α
obligation␈αon␈α
men,␈α
including␈α
men␈αof
␈↓ α∧␈↓↓science,␈α
to␈α
exempt␈α
life␈α
itself␈αfrom␈α
the␈α
madness␈α
of␈α
treating␈αeverything␈α
as␈α
an␈α
object,␈α
a␈αsufficient␈α
reason,
␈↓ α∧␈↓↓and␈αone␈αthat␈αdoes␈αnot␈αeven␈αhave␈αto␈αbe␈αspoken?␈α Why␈αdoes␈αit␈αhave␈αto␈αbe␈αexplained?␈α It␈αwould␈αappear
␈↓ α∧␈↓↓that␈α
even␈α
the␈α
noblest␈α
acts␈αof␈α
the␈α
most␈α
well-meaning␈α
people␈αare␈α
poisoned␈α
by␈α
the␈α
corrosive␈α
climate␈αof
␈↓ α∧␈↓↓values␈αof␈αour␈αtime."␈↓␈αIs␈αWeizenbaum␈αagainst␈αall␈αexperimental␈αbiology␈αor␈αeven␈αall␈αexperiments␈αwith
␈↓ α∧␈↓DNA?␈α I␈αwould␈αhesitate␈αto␈α
conclude␈αso␈αfrom␈αthis␈αquote;␈α
he␈αmay␈αsay␈αthe␈αdirect␈αopposite␈α
somewhere
␈↓ α∧␈↓else.

␈↓ α∧␈↓␈↓ αT␈↓↓"Those␈αwho␈α
know␈αwho␈α
and␈αwhat␈αthey␈α
are␈αdo␈α
not␈αneed␈α
to␈αask␈αwhat␈α
they␈αshould␈α
do."␈↓␈α-␈αpage␈α
273.
␈↓ α∧␈↓Let␈α
me␈α
assure␈α
the␈α
reader␈α
that␈α
there␈α
is␈α
nothing␈α
in␈α
the␈α
book␈α
that␈α
offers␈α
any␈α
way␈α
to␈α
interpret␈αthis
␈↓ α∧␈↓pomposity.␈α The␈αmenace␈αof␈αsuch␈αgrandiloquent␈αprecepts␈α
is␈αthat␈αthey␈αrequire␈αa␈αpriesthood␈αto␈α
apply
␈↓ α∧␈↓them␈α∞to␈α
particular␈α∞cases,␈α
and␈α∞volunteer␈α
priests␈α∞quickly␈α
crystallize␈α∞around␈α
any␈α∞potential␈α∞center␈α
of
␈↓ α∧␈↓power.

␈↓ α∧␈↓␈↓ αT␈↓↓"An␈α
individual␈α
is␈α
dehumanized␈α
whenever␈α
he␈α
is␈α
treated␈α
as␈α
less␈α
than␈α
a␈α
whole␈α
person"␈↓.␈α
-␈αpage
␈↓ α∧␈↓266.  This is also subject to priestly interpretation as in the encounter group movement.

␈↓ α∧␈↓α␈↓ ∧[BUT HERE'S A TRY AT SUMMARIZING:

␈↓ α∧␈↓␈↓ αTAs␈α∞these␈α
inconsistent␈α∞passages␈α
show,␈α∞it␈α
isn't␈α∞easy␈α
to␈α∞determine␈α
Weizenbaum's␈α∞position,␈α
but
␈↓ α∧␈↓the following seem to be the book's main points:


␈↓ α∧␈↓α␈↓ αT1. Computers cannot be made to reason as powerfully as humans.

␈↓ α∧␈↓␈↓ αTThis␈α
is␈α∞supported␈α
by␈α
quoting␈α∞overoptimistic␈α
predictions␈α
by␈α∞computer␈α
scientists␈α∞and␈α
giving
␈↓ α∧␈↓examples␈α
of␈α
non-verbal␈α
human␈α
communication.␈α
 However,␈α
Weizenbaum␈α
doesn't␈α
name␈αany␈α
specific
␈↓ α∧␈↓task␈α∞that␈α∞computers␈α∞cannot␈α∞carry␈α∞out,␈α∞because␈α∞he␈α∞wishes␈α∞␈↓↓"to␈α∞avoid␈α∞the␈α∂unnecessary,␈α∞interminable,
␈↓ α∧␈↓↓and␈αultimately␈αsterile␈αexercise␈αof␈αmaking␈αa␈αcatalogue␈αof␈αwhat␈αcomputers␈αwill␈αand␈αwill␈αnot␈αbe␈αable␈αto
␈↓ α∧␈↓↓do,␈α∪either␈α∩here␈α∪and␈α∪now␈α∩or␈α∪ever"␈↓.␈α∪ It␈α∩is␈α∪also␈α∩stated␈α∪that␈α∪human␈α∩and␈α∪machine␈α∪reasoning␈α∩are
␈↓ α∧␈↓incomparable and that the sensory experience of a human is essential for human reasoning.




␈↓ α∧␈↓␈↓ ε|2␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓α␈↓ αT2. There are tasks that computers should not be programmed to do.

␈↓ α∧␈↓␈↓ αTSome␈α
are␈α
tasks␈αWeizenbaum␈α
thinks␈α
shouldn't␈αbe␈α
done␈α
at␈αall␈α
-␈α
mostly␈αfor␈α
new␈α
left␈αreasons.
␈↓ α∧␈↓One␈αmay␈αquarrel␈α
with␈αhis␈αpolitics,␈αand␈α
I␈αdo,␈αbut␈αobviously␈α
computers␈αshouldn't␈αdo␈αwhat␈α
shouldn't
␈↓ α∧␈↓be␈α
done.␈α
 However,␈α
Weizenbaum␈α
also␈α
objects␈α
to␈α
computer␈α
hookups␈α
to␈α
animal␈α
brains␈αand␈α
computer
␈↓ α∧␈↓conducted␈α∀psychiatric␈α∪interviews.␈α∀ As␈α∪to␈α∀the␈α∪former,␈α∀I␈α∪couldn't␈α∀tell␈α∪whether␈α∀he␈α∪is␈α∀an␈α∪anti-
␈↓ α∧␈↓vivisectionist,␈αbut␈αhe␈αseems␈αto␈αhave␈αadditional␈αreasons␈αfor␈αcalling␈αthem␈α"obscene".␈α The␈αobjection
␈↓ α∧␈↓to␈α
computers␈α
doing␈α
psychiatric␈α
interviews␈α
also␈αhas␈α
a␈α
component␈α
beyond␈α
the␈α
conviction␈α
that␈αthey
␈↓ α∧␈↓would␈αnecessarily␈αdo␈αit␈αbadly.␈α Thus␈αhe␈αsays,␈α␈↓↓"What␈αcan␈αthe␈αpsychiatrist's␈αimage␈αof␈αhis␈α
patient␈αbe
␈↓ α∧␈↓↓when␈αhe␈αsees␈αhimself,␈αas␈αa␈αtherapist,␈αnot␈αas␈αan␈αengaged␈αhuman␈αbeing␈αacting␈αas␈αa␈αhealer,␈αbut␈αas␈αan
␈↓ α∧␈↓↓information␈αprocessor␈αfollowing␈αrules,␈αetc.?"␈↓␈αThis␈αseems␈αlike␈αthe␈αrenaissance␈αera␈αreligious␈αobjections
␈↓ α∧␈↓to␈α∂dissecting␈α∂the␈α∞human␈α∂body␈α∂that␈α∂came␈α∞up␈α∂when␈α∂science␈α∞revived.␈α∂ Even␈α∂the␈α∂Popes␈α∞eventually
␈↓ α∧␈↓convinced␈αthemselves␈αthat␈αregarding␈α
the␈αbody␈αas␈αa␈αmachine␈α
for␈αscientific␈αor␈αmedical␈αpurposes␈α
was
␈↓ α∧␈↓quite␈αcompatible␈αwith␈αregarding␈αit␈α
as␈αthe␈αtemple␈αof␈αthe␈α
soul.␈α Recently␈αthey␈αhave␈αtaken␈α
the␈αsame
␈↓ α∧␈↓view of studying mental mechanisms for scientific or psychiatric purposes.


␈↓ α∧␈↓α␈↓ αT3. Science has led people to a wrong view of the world and of life.

␈↓ α∧␈↓␈↓ αTThe␈αview␈αis␈αcharacterized␈αas␈αmechanistic,␈αand␈αthe␈αexample␈αof␈αclockwork␈αis␈αgiven.␈α (It␈αseems
␈↓ α∧␈↓strange␈αfor␈αa␈αcomputer␈αscientist␈αto␈αgive␈αthis␈αexample,␈αbecause␈αthe␈αadvance␈αof␈αthe␈αcomputer␈αmodel
␈↓ α∧␈↓over␈α∀older␈α∃mechanistic␈α∀models␈α∀is␈α∃that␈α∀computers␈α∀can␈α∃and␈α∀clockwork␈α∀can't␈α∃make␈α∀decisions.)
␈↓ α∧␈↓Apparently␈αanalysis␈αof␈αa␈αliving␈αsystem␈αas␈αcomposed␈αof␈αinteracting␈αparts␈αrather␈αthan␈αtreating␈αit␈αas
␈↓ α∧␈↓an unanalyzed whole is bad.


␈↓ α∧␈↓α␈↓ αT4. Science is not the sole or even main source of reliable knowledge.

␈↓ α∧␈↓␈↓ αTHowever,␈α⊂he␈α⊂doesn't␈α⊃propose␈α⊂any␈α⊂other␈α⊃sources␈α⊂of␈α⊂knowledge␈α⊂or␈α⊃say␈α⊂what␈α⊂the␈α⊃limits␈α⊂of
␈↓ α∧␈↓scientific knowledge is except to characterize certain thoughts as "obscene".


␈↓ α∧␈↓α␈↓ αT5. Certain people and institutions are bad.

␈↓ α∧␈↓␈↓ αTThese␈α∂include␈α∂the␈α∂Department␈α∂of␈α∂"Defense"␈α∂(sic),␈α∂␈↓↓Psychology␈α∂Today␈↓,␈α∂the␈α∂␈↓↓New␈α∂York␈α∞Times␈↓
␈↓ α∧␈↓Data␈α
Bank,␈α
compulsive␈α
computer␈α
programmers,␈α
Kenneth␈α
Colby,␈α
Marvin␈α
Minsky,␈α∞Roger␈α
Schank,
␈↓ α∧␈↓Allen␈αNewell,␈αHerbert␈αSimon,␈αJ.W.␈α Forrester,␈αEdward␈αFredkin,␈αB.F.␈αSkinner,␈αWarren␈αMcCulloch
␈↓ α∧␈↓(until he was old), Laplace and Leibniz.

␈↓ α∧␈↓α␈↓ ¬aTHE ELIZA EXAMPLE

␈↓ α∧␈↓␈↓ αTPerhaps␈α
the␈α∞most␈α
interesting␈α∞part␈α
of␈α∞the␈α
book␈α∞is␈α
the␈α∞account␈α
of␈α∞his␈α
own␈α∞program␈α
ELIZA
␈↓ α∧␈↓that␈α∩parodies␈α∩Rogerian␈α⊃non-directive␈α∩psychotherapy␈α∩and␈α∩his␈α⊃anecdotal␈α∩account␈α∩of␈α∩how␈α⊃some
␈↓ α∧␈↓people␈αascribe␈αintelligence␈αand␈αpersonality␈αto␈αit.␈α In␈αmy␈αopinion,␈αit␈αis␈αquite␈αnatural␈αfor␈αpeople␈αwho
␈↓ α∧␈↓don't␈αunderstand␈αthe␈αnotion␈αof␈αalgorithm␈αto␈αimagine␈αthat␈αa␈αcomputer␈αcomputes␈αanalogously␈αto␈αthe
␈↓ α∧␈↓way␈αa␈αhuman␈αreasons.␈α This␈αleads␈αto␈αthe␈αidea␈αthat␈αaccurate␈αcomputation␈αentails␈αcorrect␈αreasoning
␈↓ α∧␈↓and␈αeven␈α
to␈αthe␈αidea␈α
that␈αcomputer␈αmalfunctions␈α
are␈αanalogous␈αto␈α
human␈αneuroses␈αand␈α
psychoses.


␈↓ α∧␈↓␈↓ ε|3␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓Actually,␈αprogramming␈αa␈αcomputer␈αto␈αdraw␈αinteresting␈αconclusions␈αfrom␈αpremises␈αis␈αvery␈αdifficult
␈↓ α∧␈↓and␈α
only␈α
limited␈α
success␈α∞has␈α
been␈α
attained.␈α
 However,␈α∞the␈α
effect␈α
of␈α
these␈α∞natural␈α
misconceptions
␈↓ α∧␈↓shouldn't␈α⊂be␈α⊂exaggerated;␈α⊂people␈α⊂readily␈α⊂understand␈α⊂the␈α⊂truth␈α⊂when␈α⊂it␈α⊂is␈α⊂explained,␈α⊂especially
␈↓ α∧␈↓when␈α⊃it␈α⊃applies␈α⊃to␈α⊃a␈α⊂matter␈α⊃that␈α⊃concerns␈α⊃them.␈α⊃ In␈α⊂particular,␈α⊃when␈α⊃an␈α⊃executive␈α⊃excuses␈α⊂a
␈↓ α∧␈↓mistake by saying that he placed excessive faith in a computer, a certain skepticism is called for.

␈↓ α∧␈↓␈↓ αTColby's␈α
(1973)␈α
study␈α
is␈α
interesting␈α
in␈α
this␈α
connection,␈α
but␈α
the␈α
interpretation␈α
below␈α∞is␈α
mine.
␈↓ α∧␈↓Colby␈α∞had␈α∞psychiatrists␈α
interview␈α∞patients␈α∞over␈α∞a␈α
teletype␈α∞line␈α∞and␈α
also␈α∞had␈α∞them␈α∞interview␈α
his
␈↓ α∧␈↓PARRY␈α
program␈α
that␈α
simulates␈α
a␈α
paranoid.␈α∞ Other␈α
psychiatrists␈α
were␈α
asked␈α
to␈α
decide␈α∞from␈α
the
␈↓ α∧␈↓transcripts␈αwhether␈αthe␈α
interview␈αwas␈αwith␈α
a␈αman␈αor␈αwith␈α
a␈αprogram,␈αand␈α
they␈αdid␈αno␈αbetter␈α
than
␈↓ α∧␈↓chance.␈α
 However,␈α
since␈αPARRY␈α
is␈α
incapable␈α
of␈αthe␈α
simplest␈α
causal␈αreasoning,␈α
if␈α
you␈α
ask,␈α"How
␈↓ α∧␈↓do␈αyou␈αknow␈αthe␈αpeople␈αfollowing␈αyou␈αare␈αMafia"␈αand␈αget␈αa␈αreply␈αthat␈αthey␈αlook␈αlike␈αItalians,␈α
this
␈↓ α∧␈↓must␈αbe␈αa␈αman␈αnot␈αPARRY.␈α Curiously,␈αit␈αis␈αeasier␈αto␈αimitate␈α(well␈αenough␈αto␈αfool␈αa␈αpsychiatrist)
␈↓ α∧␈↓the␈α∩emotional␈α∪side␈α∩of␈α∩a␈α∪man␈α∩than␈α∩his␈α∪intellectual␈α∩side.␈α∩ Probably␈α∪the␈α∩subjects␈α∪expected␈α∩the
␈↓ α∧␈↓machine␈α∂to␈α∞have␈α∂more␈α∂logical␈α∞ability,␈α∂and␈α∞this␈α∂expectation␈α∂contributed␈α∞to␈α∂their␈α∂mistakes.␈α∞ Alas,
␈↓ α∧␈↓random selection from the directory of the Association for Computing Machinery did no better.

␈↓ α∧␈↓␈↓ αTIt␈α∂seems␈α⊂to␈α∂me␈α⊂that␈α∂ELIZA␈α⊂and␈α∂PARRY␈α⊂show␈α∂only␈α⊂that␈α∂people,␈α⊂including␈α∂psychiatrists,
␈↓ α∧␈↓often␈αhave␈αto␈αdraw␈αconclusions␈αon␈αslight␈αevidence,␈αand␈αare␈αtherefore␈αeasily␈αfooled.␈α If␈αI␈αam␈αright,
␈↓ α∧␈↓two sentences of instruction would allow them to do better.

␈↓ α∧␈↓␈↓ αTIn his 1966 paper on ELIZA (incorrectly cited as 1965), Weizenbaum writes,

␈↓ α∧␈↓␈↓ αT␈↓↓"One␈αgoal␈αfor␈αan␈αaugmented␈αELIZA␈αprogram␈α
is␈αthus␈αa␈αsystem␈αwhich␈αalready␈αhas␈αaccess␈α
to␈αa
␈↓ α∧␈↓↓store␈α⊂of␈α⊂information␈α⊂about␈α⊂some␈α⊂aspect␈α⊂of␈α⊂the␈α⊂real␈α⊂world␈α⊂and␈α⊂which,␈α⊂by␈α⊂means␈α⊂of␈α⊂conversational
␈↓ α∧␈↓↓interaction␈αwith␈αpeople,␈αcan␈αreveal␈α
both␈αwhat␈αit␈αknows,␈αi.e.␈α
behave␈αas␈αan␈αinformation␈αretrieval␈α
system,
␈↓ α∧␈↓↓and␈α⊃where␈α⊂its␈α⊃knowledge␈α⊂ends␈α⊃and␈α⊃needs␈α⊂to␈α⊃be␈α⊂augmented.␈α⊃ Hopefully␈α⊂the␈α⊃augmentation␈α⊃of␈α⊂its
␈↓ α∧␈↓↓knowledge␈α⊃will␈α⊂also␈α⊃be␈α⊃a␈α⊂direct␈α⊃consequence␈α⊃of␈α⊂its␈α⊃conversational␈α⊃experience.␈α⊂ It␈α⊃is␈α⊃precisely␈α⊂the
␈↓ α∧␈↓↓prospect␈α∂that␈α∞such␈α∂a␈α∂program␈α∞will␈α∂converse␈α∞with␈α∂many␈α∂people␈α∞and␈α∂learn␈α∞something␈α∂from␈α∂each␈α∞of
␈↓ α∧␈↓↓them␈α⊃which␈α⊃leads␈α⊃to␈α⊃the␈α⊃hope␈α⊃that␈α⊃it␈α⊃will␈α⊃prove␈α⊃an␈α⊃interesting␈α⊃and␈α⊃even␈α⊃useful␈α⊂conversational
␈↓ α∧␈↓↓partner."␈↓␈αToo␈αbad␈αhe␈αdidn't␈αsuccessfully␈αpursue␈αthis␈αgoal;␈αno-one␈αelse␈αhas␈αdone␈αso␈αeither.␈α I␈αthink
␈↓ α∧␈↓success␈α⊂would␈α∂have␈α⊂required␈α⊂a␈α∂better␈α⊂understanding␈α∂of␈α⊂formalization␈α⊂than␈α∂is␈α⊂exhibited␈α⊂in␈α∂the
␈↓ α∧␈↓book.

␈↓ α∧␈↓α␈↓ ∧?WHAT DOES HE SAY ABOUT COMPUTERS?

␈↓ α∧␈↓␈↓ αTWhile␈α⊃Weizenbaum's␈α⊂main␈α⊃conclusions␈α⊃concern␈α⊂science␈α⊃in␈α⊃general␈α⊂and␈α⊃are␈α⊃moralistic␈α⊂in
␈↓ α∧␈↓character, some of his remarks about computer science and AI are worthy of comment.

␈↓ α∧␈↓␈↓ αT1.␈α∂ He␈α⊂concludes␈α∂that␈α⊂since␈α∂a␈α⊂computer␈α∂cannot␈α⊂have␈α∂the␈α⊂experience␈α∂of␈α⊂a␈α∂man,␈α⊂it␈α∂cannot
␈↓ α∧␈↓understand␈αa␈αman.␈α There␈αare␈αthree␈αpoints␈αto␈αbe␈αmade␈αin␈αreply.␈α First,␈αhumans␈αshare␈αeach␈αother's
␈↓ α∧␈↓experiences␈α∞and␈α∞those␈α∞of␈α∞machines␈α∞or␈α∞animals␈α
only␈α∞to␈α∞a␈α∞limited␈α∞extent.␈α∞ In␈α∞particular,␈α∞men␈α
and
␈↓ α∧␈↓women␈α
have␈α
different␈α
experiences.␈α
 Nevertheless,␈α
it␈α
is␈α
common␈α
in␈α
literature␈α
for␈α
a␈α
good␈α
writer␈αto
␈↓ α∧␈↓show␈α
greater␈αunderstanding␈α
of␈αthe␈α
experience␈αof␈α
the␈α
opposite␈αsex␈α
than␈αa␈α
poorer␈αwriter␈α
of␈αthat␈α
sex.
␈↓ α∧␈↓Second,␈αthe␈αnotion␈αof␈αexperience␈αis␈αpoorly␈αunderstood;␈αif␈αwe␈αunderstood␈αit␈αbetter,␈αwe␈αcould␈αreason
␈↓ α∧␈↓about␈α∞whether␈α∞a␈α∞machine␈α∞could␈α∞have␈α∞a␈α∞simulated␈α∞or␈α∞vicarious␈α∞experience␈α∞normally␈α∞confined␈α∞to
␈↓ α∧␈↓humans.␈α Third,␈αwhat␈α
we␈αmean␈αby␈αunderstanding␈α
is␈αpoorly␈αunderstood,␈αso␈α
we␈αdon't␈αyet␈αknow␈α
how
␈↓ α∧␈↓to define whether a machine understands something or not.

␈↓ α∧␈↓␈↓ ε|4␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓␈↓ αT2.␈α∀Like␈α∀his␈α∀predecessor␈α∀critics␈α∪of␈α∀artificial␈α∀intelligence,␈α∀Taube,␈α∀Dreyfus␈α∀and␈α∪Lighthill,
␈↓ α∧␈↓Weizenbaum␈α
is␈αimpatient,␈α
concluding␈αthat␈α
if␈α
the␈αproblem␈α
hasn't␈αbeen␈α
solved␈αin␈α
twenty␈α
years,␈αwe
␈↓ α∧␈↓should␈α
give␈αup.␈α
 Genetics␈α
took␈αabout␈α
a␈α
century␈αto␈α
go␈α
from␈αMendel␈α
to␈α
the␈αgenetic␈α
code␈αfor␈α
proteins,
␈↓ α∧␈↓and␈α⊂still␈α⊂has␈α⊂a␈α⊂long␈α⊂way␈α⊂to␈α⊂go␈α∂before␈α⊂we␈α⊂will␈α⊂fully␈α⊂understand␈α⊂the␈α⊂genetics␈α⊂and␈α⊂evolution␈α∂of
␈↓ α∧␈↓intelligence␈α
and␈α
behavior.␈α
 Artificial␈α
intelligence␈α
may␈α∞be␈α
just␈α
as␈α
difficult.␈α
 My␈α
current␈α∞answer␈α
to
␈↓ α∧␈↓the␈α∞question␈α∞of␈α∞when␈α∂machines␈α∞will␈α∞reach␈α∞human-level␈α∂intelligence␈α∞is␈α∞that␈α∞a␈α∂precise␈α∞calculation
␈↓ α∧␈↓shows␈αthat␈αwe␈α
are␈αbetween␈α1.7␈αand␈α
3.1␈αEinsteins␈αand␈α.3␈α
Manhattan␈αProjects␈αaway␈αfrom␈α
the␈αgoal.
␈↓ α∧␈↓However,␈α⊃the␈α⊃current␈α⊃research␈α⊃is␈α⊃producing␈α⊂the␈α⊃information␈α⊃on␈α⊃which␈α⊃the␈α⊃Einstein␈α⊃will␈α⊂base
␈↓ α∧␈↓himself and is producing useful capabilities all the time.

␈↓ α∧␈↓␈↓ αT3.␈α⊂The␈α⊃book␈α⊂nowhere␈α⊃distinguishes␈α⊂between␈α⊃formalizability␈α⊂and␈α⊃computer␈α⊂simulatability.
␈↓ α∧␈↓Formalizations␈αin␈αlogic␈αcan␈αallow␈αanswering␈αquestions␈αthat␈αcannot␈αbe␈αanswered␈αby␈αsimulation,␈αe.g.
␈↓ α∧␈↓"would␈α∞it␈α∞ever␈α∂do␈α∞so-and-so".␈α∞ Reasoning␈α∂in␈α∞a␈α∞formal␈α∞system␈α∂can␈α∞often␈α∞use␈α∂partial␈α∞information
␈↓ α∧␈↓insufficient␈α∀for␈α∀simulation␈α∃to␈α∀answer␈α∀such␈α∃questions.␈α∀ In␈α∀fact,␈α∃a␈α∀simulation␈α∀program␈α∃for␈α∀a
␈↓ α∧␈↓phenomenon␈α
is␈α
only␈α
one␈α
of␈α
many␈α
possible␈α
formalizations␈α
of␈α
the␈α
phenomenon.␈α
 This␈α
review␈α
isn't
␈↓ α∧␈↓the␈α∂place␈α∂for␈α∂a␈α∂full␈α∂explanation␈α∂of␈α∂the␈α∂relations␈α∂between␈α∂these␈α∂concepts.␈α∂ Like␈α⊂␈↓↓Punch␈↓'s␈α∂famous
␈↓ α∧␈↓curate's egg, the book is good in parts.  Thus it raises the following interesting issues:

␈↓ α∧␈↓␈↓ αT1.␈αWhat␈αwould␈αit␈αmean␈αfor␈αa␈αcomputer␈αto␈αhope␈αor␈αbe␈αdesperate␈αfor␈αlove?␈α Answers␈αto␈αthese
␈↓ α∧␈↓questions␈αdepend␈αon␈αbeing␈αable␈αto␈αformalize␈α(not␈αsimulate)␈αthe␈αphenomena␈αin␈αquestion.␈α My␈α
guess
␈↓ α∧␈↓is␈αthat␈αadding␈α
a␈αnotion␈αof␈α
hope␈αto␈αan␈αaxiomatization␈α
of␈αbelief␈αand␈α
wanting␈αmight␈αnot␈αbe␈α
difficult.
␈↓ α∧␈↓The study of ␈↓↓propositional attitudes␈↓ in philosophical logic points in that direction.

␈↓ α∧␈↓␈↓ αT2.␈α∀Do␈α∀differences␈α∀in␈α∀experience␈α∀make␈α∀human␈α∀and␈α∀machine␈α∀intelligence␈α∀necessarily␈α∀so
␈↓ α∧␈↓different␈αthat␈αit␈αis␈αmeaningless␈αto␈αask␈αwhether␈αa␈αmachine␈αcan␈αbe␈αmore␈αintelligent␈αthan␈αa␈α
machine?
␈↓ α∧␈↓My␈α
opinion␈α
is␈α
that␈α
comparison␈α
will␈α
turn␈α∞out␈α
to␈α
be␈α
meaningful.␈α
 After␈α
all,␈α
most␈α
people␈α∞have␈α
not
␈↓ α∧␈↓doubt␈αthat␈αhumans␈αare␈αmore␈αintelligent␈αthan␈αturkeys.␈α Weizenbaum's␈αexamples␈αof␈αthe␈αdependence
␈↓ α∧␈↓of␈α↔human␈α⊗intelligence␈α↔on␈α⊗sensory␈α↔abilities␈α⊗seem␈α↔even␈α⊗refutable,␈α↔because␈α⊗we␈α↔recognize␈α⊗no
␈↓ α∧␈↓fundamental␈α
difference␈α
in␈α
humanness␈αin␈α
people␈α
who␈α
are␈αseverely␈α
handicapped␈α
sensorily,␈α
e.g.␈αthe
␈↓ α∧␈↓deaf, dumb and blind or paraplegics.

␈↓ α∧␈↓α␈↓ α+IN DEFENSE OF THE UNJUSTLY ATTACKED - SOME OF WHOM ARE INNOCENT

␈↓ α∧␈↓␈↓ αTHere␈α∞are␈α∞defenses␈α∞of␈α∞Weizenbaum's␈α∞targets.␈α∞ They␈α∞are␈α∞not␈α∞guaranteed␈α∞to␈α∞entirely␈α∞suit␈α∞the
␈↓ α∧␈↓defendees.

␈↓ α∧␈↓␈↓ αTWeizenbaum's␈αconjecture␈αthat␈αthe␈α
Defense␈αDepartment␈αsupports␈αspeech␈αrecognition␈α
research
␈↓ α∧␈↓in␈α⊃order␈α⊃to␈α⊃be␈α⊃able␈α⊃to␈α⊃snoop␈α⊂on␈α⊃telephone␈α⊃conversations␈α⊃is␈α⊃biased,␈α⊃baseless,␈α⊃false,␈α⊃and␈α⊂seems
␈↓ α∧␈↓motivated␈α∂by␈α∂political␈α⊂malice.␈α∂ The␈α∂committee␈α∂of␈α⊂scientists␈α∂that␈α∂proposed␈α∂the␈α⊂project␈α∂advanced
␈↓ α∧␈↓quite␈α
different␈αconsiderations,␈α
and␈αthe␈α
high␈αofficials␈α
who␈αmade␈α
the␈αfinal␈α
decisions␈αare␈α
not␈αogres.
␈↓ α∧␈↓Anyway␈α≤their␈α≠other␈α≤responsibilities␈α≠leave␈α≤them␈α≠no␈α≤time␈α≠for␈α≤complicated␈α≤and␈α≠devious
␈↓ α∧␈↓considerations.␈α∞ I␈α
put␈α∞this␈α
one␈α∞first,␈α
because␈α∞I␈α
think␈α∞the␈α
failure␈α∞of␈α
many␈α∞scientists␈α
to␈α∞defend␈α
the
␈↓ α∧␈↓Defense␈αDepartment␈αagainst␈α
attacks␈αthey␈αknow␈α
are␈αunjustified,␈αis␈α
unjust␈αin␈αitself,␈αand␈α
furthermore
␈↓ α∧␈↓has harmed the country.

␈↓ α∧␈↓␈↓ αTWeizenbaum␈α
doubts␈α
that␈α
computer␈α
speech␈α
recognition␈α
will␈α
have␈α
cost-effective␈αapplications
␈↓ α∧␈↓beyond␈α∞snooping␈α∞on␈α∞phone␈α∂conversations.␈α∞ He␈α∞also␈α∞says,␈α∞␈↓↓"There␈α∂is␈α∞no␈α∞question␈α∞in␈α∞my␈α∂mind␈α∞that


␈↓ α∧␈↓␈↓ ε|5␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓↓there␈αis␈αno␈αpressing␈αhuman␈αproblem␈αthat␈αwill␈αbe␈αmore␈αeasily␈αsolved␈αbecause␈αsuch␈αmachines␈α
exist"␈↓.␈α I
␈↓ α∧␈↓worry␈αmore␈α
about␈αwhether␈αthe␈α
programs␈αcan␈αbe␈α
made␈αto␈αwork␈α
before␈αthe␈αsponsor␈α
loses␈αpatience.
␈↓ α∧␈↓Once␈α
they␈α
work,␈α
costs␈α
will␈α
come␈α
down.␈α
 Winograd␈α
pointed␈α
out␈α
to␈α
me␈α
that␈α
many␈α
possible␈α
household
␈↓ α∧␈↓applications␈α∞of␈α∞computers␈α∞may␈α∞not␈α∂be␈α∞feasible␈α∞without␈α∞some␈α∞computer␈α∞speech␈α∂recognition.␈α∞ One
␈↓ α∧␈↓needs␈αto␈αthink␈αboth␈αabout␈αhow␈αto␈αsolve␈αrecognized␈αproblems␈αand␈αabout␈αopportunities␈αto␈αput␈αnew
␈↓ α∧␈↓technological␈α↔possibilities␈α↔to␈α⊗good␈α↔use.␈α↔ The␈α⊗telephone␈α↔was␈α↔not␈α⊗invented␈α↔by␈α↔a␈α⊗committee
␈↓ α∧␈↓considering already identified problems of communication.

␈↓ α∧␈↓␈↓ αTReferring␈α
to␈α
␈↓↓Psychology␈α
Today␈↓␈α
as␈α
a␈αcafeteria␈α
simply␈α
excites␈α
the␈α
snobbery␈α
of␈α
those␈αwho␈α
would
␈↓ α∧␈↓like␈α
to␈α
consider␈α
their␈α
psychological␈α
knowledge␈α
to␈α∞be␈α
above␈α
the␈α
popular␈α
level.␈α
 So␈α
far␈α
as␈α∞I␈α
know,
␈↓ α∧␈↓professional␈α
and␈αacademic␈α
psychologists␈αwelcome␈α
the␈αopportunity␈α
offered␈αby␈α
␈↓↓Psychology␈α
Today␈↓␈αto
␈↓ α∧␈↓explain␈αtheir␈αideas␈αto␈αa␈αwide␈αpublic.␈α They␈αmight␈αeven␈αbuy␈αa␈αcut-down␈αversion␈αof␈αWeizenbaum's
␈↓ α∧␈↓book␈α∞if␈α
he␈α∞asks␈α
them␈α∞nicely.␈α
 Hmm,␈α∞they␈α
might␈α∞even␈α
buy␈α∞this␈α
review.␈α∞ This␈α
was␈α∞just␈α∞a␈α
passing
␈↓ α∧␈↓reference,␈α∂but␈α∂once␈α∂the␈α∂reader␈α∂has␈α∂been␈α∞alerted␈α∂to␈α∂moral␈α∂issues,␈α∂literary␈α∂morality␈α∂also␈α∂comes␈α∞to
␈↓ α∧␈↓mind.

␈↓ α∧␈↓␈↓ αTWeizenbaum␈α
has␈α
invented␈αa␈α
␈↓↓New␈α
York␈αTimes␈α
Data␈α
Bank␈↓␈αdifferent␈α
from␈α
the␈α
one␈αoperated
␈↓ α∧␈↓by␈α
the␈α
␈↓↓New␈α
York␈α
Times␈↓␈α
-␈α
and␈α
possibly␈α
better.␈α
 The␈α
real␈α
one␈α
stores␈α
abstracts␈α
written␈α∞by␈α
humans
␈↓ α∧␈↓and␈αdoesn't␈αuse␈αthe␈αtapes␈αintended␈αfor␈αtypesetting␈αmachines.␈α As␈αa␈αresult␈αthe␈αuser␈αhas␈αaccess␈αonly
␈↓ α∧␈↓to␈α
abstracts␈α
and␈αcannot␈α
search␈α
on␈α
features␈αof␈α
the␈α
stories␈αthemselves,␈α
i.e.␈α
 he␈α
is␈αat␈α
the␈α
mercy␈αof␈α
what
␈↓ α∧␈↓the abstractors thought was important at the time.

␈↓ α∧␈↓␈↓ αTUsing␈α⊂computer␈α⊃programs␈α⊂as␈α⊂psychotherapists,␈α⊃as␈α⊂Colby␈α⊂proposed,␈α⊃would␈α⊂be␈α⊂moral␈α⊃if␈α⊂it
␈↓ α∧␈↓would␈α∂cure␈α∂people.␈α∂ Unfortunately,␈α∂computer␈α∂science␈α∞isn't␈α∂up␈α∂to␈α∂it,␈α∂and␈α∂maybe␈α∂the␈α∞psychiatrists
␈↓ α∧␈↓aren't either.

␈↓ α∧␈↓␈↓ αTI␈αagree␈αwith␈αMinsky␈αin␈αcriticizing␈αthe␈αreluctance␈αof␈αart␈αtheorists␈αto␈αdevelop␈αformal␈αtheories.
␈↓ α∧␈↓George␈α
Birkhoff's␈α∞formal␈α
theory␈α∞was␈α
probably␈α∞wrong,␈α
but␈α∞he␈α
shouldn't␈α∞have␈α
been␈α∞criticized␈α
for
␈↓ α∧␈↓trying.␈α⊂ The␈α⊂problem␈α⊂seems␈α⊂very␈α⊂difficult␈α⊂to␈α⊂me,␈α⊂and␈α⊂I␈α⊂have␈α⊂made␈α⊂no␈α⊂significant␈α⊂progress␈α⊂in
␈↓ α∧␈↓responding␈α
to␈αa␈α
challenge␈α
from␈αArthur␈α
Koestler␈αto␈α
tell␈α
how␈αa␈α
computer␈αprogram␈α
might␈α
make␈αor
␈↓ α∧␈↓even recognize jokes.  Perhaps some reader of this review might have more success.

␈↓ α∧␈↓␈↓ αTThere␈αis␈αa␈αwhole␈αchapter␈αattacking␈α"compulsive␈αcomputer␈αprogrammers"␈αor␈α"hackers".␈α This
␈↓ α∧␈↓mythical␈αbeast␈αlives␈αin␈αthe␈αcomputer␈αlaboratory,␈αis␈αan␈αexpert␈αon␈αall␈αthe␈αins␈αand␈αouts␈αof␈αthe␈αtime-
␈↓ α∧␈↓sharing␈αsystem,␈αelaborates␈αthe␈αtime-sharing␈αsystem␈αwith␈αarcane␈αfeatures␈αthat␈αhe␈αnever␈αdocuments,
␈↓ α∧␈↓and␈α∞is␈α∞always␈α∞changing␈α∞the␈α∂system␈α∞before␈α∞he␈α∞even␈α∞fixes␈α∂the␈α∞bugs␈α∞in␈α∞the␈α∞previous␈α∂version.␈α∞ All
␈↓ α∧␈↓these␈α∞vices␈α∞exist,␈α∞but␈α∞I␈α∞can't␈α∞think␈α∞of␈α∞any␈α∞individual␈α∞who␈α∞combines␈α∞them,␈α∞and␈α∂people␈α∞generally
␈↓ α∧␈↓outgrow␈α∞them.␈α∞ As␈α∞a␈α∞laboratory␈α
director,␈α∞I␈α∞have␈α∞to␈α∞protect␈α
the␈α∞interests␈α∞of␈α∞people␈α∞who␈α
program
␈↓ α∧␈↓only␈α
part␈α
time␈α
against␈αtendencies␈α
to␈α
over-complicate␈α
the␈α
facilities.␈α People␈α
who␈α
spend␈α
all␈αtheir␈α
time
␈↓ α∧␈↓programming␈αand␈αwho␈αexchange␈αinformation␈αby␈αword␈αof␈αmouth␈αsometimes␈αhave␈αto␈αbe␈αpressed␈αto
␈↓ α∧␈↓make␈α⊂proper␈α⊂writeups.␈α⊂ The␈α⊂other␈α⊂side␈α⊂of␈α∂the␈α⊂issue␈α⊂is␈α⊂that␈α⊂we␈α⊂professors␈α⊂of␈α⊂computer␈α∂science
␈↓ α∧␈↓sometimes␈α
lose␈α
our␈α
ability␈α∞to␈α
write␈α
actual␈α
computer␈α
programs␈α∞through␈α
lack␈α
of␈α
practice␈α∞and␈α
envy
␈↓ α∧␈↓younger␈α∞people␈α∞who␈α
can␈α∞spend␈α∞full␈α∞time␈α
in␈α∞the␈α∞laboratory.␈α∞ The␈α
phenomenon␈α∞is␈α∞well␈α∞known␈α
in
␈↓ α∧␈↓other sciences and in other human activities.

␈↓ α∧␈↓␈↓ αTWeizenbaum␈α∩attacks␈α⊃the␈α∩Yale␈α⊃computer␈α∩linguist,␈α∩Roger␈α⊃Schank,␈α∩as␈α⊃follows␈α∩-␈α∩the␈α⊃inner
␈↓ α∧␈↓quotes␈α∞are␈α∞from␈α∞Schank:␈α∞␈↓↓"What␈α∞is␈α∞contributed␈α∞when␈α∞it␈α∞is␈α∞asserted␈α∞that␈α∞'there␈α∞exists␈α∞a␈α∞conceptual


␈↓ α∧␈↓␈↓ ε|6␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓↓base␈α⊂that␈α⊂is␈α⊂interlingual,␈α⊂onto␈α⊂which␈α⊃linguistic␈α⊂structures␈α⊂in␈α⊂a␈α⊂given␈α⊂language␈α⊂map␈α⊃during␈α⊂the
␈↓ α∧␈↓↓understanding␈α
process␈αand␈α
out␈αof␈α
which␈αsuch␈α
structures␈αare␈α
created␈αduring␈α
generation␈α[of␈α
linguistic
␈↓ α∧␈↓↓utterances]'?␈α Nothing␈αat␈αall.␈α For␈αthe␈αterm␈α'conceptual␈αbase'␈αcould␈αperfectly␈αwell␈αbe␈αreplaced␈αby␈αthe
␈↓ α∧␈↓↓word␈α'something'.␈α And␈αwho␈αcould␈αargue␈αwith␈αthat␈αso-transformed␈αstatement?"␈↓␈αWeizenbaum␈αgoes␈αon
␈↓ α∧␈↓to␈αsay␈αthat␈αthe␈αreal␈αscientific␈αproblem␈α"remains␈αas␈αuntouched␈αas␈αever".␈α On␈αthe␈αnext␈αpage␈αhe␈αsays
␈↓ α∧␈↓that␈α∞unless␈α∞the␈α∞"Schank-like␈α∞scheme"␈α∞understood␈α∞the␈α
sentence␈α∞␈↓↓"Will␈α∞you␈α∞come␈α∞to␈α∞dinner␈α∞with␈α
me
␈↓ α∧␈↓↓this␈αevening?"␈↓␈αto␈αmean␈α␈↓↓"a␈αshy␈αyoung␈αman's␈αdesperate␈αlonging␈αfor␈αlove␈↓,␈αthen␈αthe␈αsense␈αin␈αwhich␈αthe
␈↓ α∧␈↓system␈α
"understands"␈α
is␈α
"about␈α
as␈α
weak␈α
as␈α
the␈α
sense␈α
in␈α
which␈α
ELIZA␈α
"understood"".␈α
 This␈α
good
␈↓ α∧␈↓example␈α
raises␈α
interesting␈α
issues␈αand␈α
seems␈α
to␈α
call␈α
for␈αsome␈α
distinctions.␈α
 Full␈α
understanding␈αof␈α
the
␈↓ α∧␈↓sentence␈αindeed␈αresults␈αin␈αknowing␈αabout␈αthe␈αyoung␈αman's␈αdesire␈αfor␈αlove,␈αbut␈αit␈αwould␈αseem␈αthat
␈↓ α∧␈↓there␈α∞is␈α∞a␈α∂useful␈α∞lesser␈α∞level␈α∂of␈α∞understanding␈α∞in␈α∂which␈α∞the␈α∞machine␈α∂would␈α∞know␈α∞only␈α∂that␈α∞he
␈↓ α∧␈↓would like her to come to dinner.

␈↓ α∧␈↓␈↓ αTContrast␈α→Weizenbaum's␈α→demanding,␈α_more-human-than-thou␈α→attitude␈α→to␈α→Schank␈α_and
␈↓ α∧␈↓Winograd␈α
with␈αhis␈α
respectful␈αand␈α
even␈αobsequious␈α
attitude␈αto␈α
Chomsky.␈α We␈α
have␈α␈↓↓"The␈α
linguist's
␈↓ α∧␈↓↓first␈α
task␈α
is␈α
therefore␈α
to␈α
write␈α
grammars,␈α
that␈α
is,␈α
sets␈α
of␈α
rules,␈α
of␈α
particular␈α
languages,␈α
grammars
␈↓ α∧␈↓↓capable␈α∂of␈α∞characterizing␈α∂all␈α∞and␈α∂only␈α∞the␈α∂grammatically␈α∞admissible␈α∂sentences␈α∞of␈α∂those␈α∞languages,
␈↓ α∧␈↓↓and␈αthen␈α
to␈αpostulate␈αprinciples␈α
from␈αwhich␈αcrucial␈α
features␈αof␈αall␈α
such␈αgrammars␈αcan␈α
be␈αdeduced.
␈↓ α∧␈↓↓That␈α
set␈α
of␈α
principles␈α
would␈α
then␈α
constitute␈α
a␈α
universal␈α
grammar.␈α
 Chomsky's␈α
hypothesis␈α
is,␈α
to␈α
put␈α
it
␈↓ α∧␈↓↓another␈α⊂way,␈α⊃that␈α⊂the␈α⊃rules␈α⊂of␈α⊂such␈α⊃a␈α⊂universal␈α⊃grammar␈α⊂would␈α⊂constitute␈α⊃a␈α⊂kind␈α⊃of␈α⊂projective
␈↓ α∧␈↓↓description␈α
of␈α
important␈α
aspects␈α
of␈α
the␈α
human␈α
mind."␈↓␈α
There␈α
is␈α
nothing␈α
here␈α
demanding␈α
that␈α
the
␈↓ α∧␈↓universal␈α⊃grammar␈α⊃take␈α⊃into␈α⊃account␈α⊃the␈α⊃young␈α∩man's␈α⊃desire␈α⊃for␈α⊃love.␈α⊃ As␈α⊃far␈α⊃as␈α⊃I␈α∩can␈α⊃see,
␈↓ α∧␈↓Chomsky is just as much a rationalist as we artificial intelligentsia.

␈↓ α∧␈↓␈↓ αTChomsky's␈αgoal␈αof␈αa␈αuniversal␈αgrammar␈αand␈αSchank's␈αgoal␈αof␈αa␈αconceptual␈αbase␈αare␈α
similar,
␈↓ α∧␈↓except␈αthat␈αSchank's␈αideas␈αare␈αfurther␈α
developed,␈αand␈αthe␈αperformance␈αof␈αhis␈α
students'␈αprograms
␈↓ α∧␈↓can␈α
be␈α
compared␈α
with␈α
reality.␈α
 I␈α
think␈α
they␈α
will␈α
require␈α
drastic␈α
revision␈α
and␈α
may␈α
not␈α
be␈α∞on␈α
the
␈↓ α∧␈↓right␈α
track␈α
at␈α
all,␈α∞but␈α
then␈α
I␈α
am␈α∞pursuing␈α
a␈α
rather␈α
different␈α∞line␈α
of␈α
research␈α
concerning␈α∞how␈α
to
␈↓ α∧␈↓represent␈αthe␈αbasic␈αfacts␈αthat␈αan␈αintelligent␈αbeing␈αmust␈αknow␈αabout␈αthe␈αworld.␈α My␈αidea␈αis␈αto␈αstart
␈↓ α∧␈↓from␈α⊗epistemology␈α↔rather␈α⊗than␈α↔from␈α⊗language,␈α⊗regarding␈α↔their␈α⊗linguistic␈α↔representation␈α⊗as
␈↓ α∧␈↓secondary.␈α
 This␈αapproach␈α
has␈αproved␈α
difficult,␈αhas␈α
attracted␈α
few␈αpractitioners,␈α
and␈αhas␈α
led␈αto␈α
few
␈↓ α∧␈↓computer programs, but I still think it's right.

␈↓ α∧␈↓␈↓ αTWeizenbaum␈α→approves␈α→of␈α→the␈α~Chomsky␈α→school's␈α→haughty␈α→attitude␈α~towards␈α→Schank,
␈↓ α∧␈↓Winograd␈αand␈αother␈αAI␈αbased␈αlanguage␈αresearchers.␈α On␈αpage␈α184,␈αhe␈αstates,␈α␈↓↓"many␈αlinguists,␈αfor
␈↓ α∧␈↓↓example,␈αNoam␈αChomsky,␈αbelieve␈αthat␈αenough␈αthinking␈αabout␈αlanguage␈αremains␈αto␈αbe␈αdone␈αto␈αoccupy
␈↓ α∧␈↓↓them␈α∩usefully␈α∩for␈α⊃yet␈α∩a␈α∩little␈α⊃while,␈α∩and␈α∩that␈α⊃any␈α∩effort␈α∩to␈α⊃convert␈α∩their␈α∩present␈α∩theories␈α⊃into
␈↓ α∧␈↓↓computer␈αmodels␈αwould,␈αif␈αattempted␈αby␈αthe␈αpeople␈αbest␈αqualified,␈αbe␈αa␈αdiversion␈αfrom␈αthe␈αmain␈αtask.
␈↓ α∧␈↓↓And they rightly see no point to spending any of their energies studying the work of the hackers."␈↓

␈↓ α∧␈↓␈↓ αTThis␈α∩brings␈α⊃the␈α∩chapter␈α∩on␈α⊃"compulsive␈α∩computer␈α∩programmers"␈α⊃alias␈α∩"hackers"␈α∩into␈α⊃a
␈↓ α∧␈↓sharper␈αfocus.␈α Chomsky's␈αlatest␈αbook␈α␈↓↓Reflections␈αon␈αLanguage␈↓␈αmakes␈αno␈αreference␈αto␈αthe␈αwork␈αof
␈↓ α∧␈↓Winograd,␈α
Schank,␈αCharniak,␈α
Wilks,␈αBobrow␈α
or␈α
William␈αWoods␈α
to␈αname␈α
only␈α
a␈αfew␈α
of␈αthose␈α
who
␈↓ α∧␈↓have␈αdeveloped␈α
large␈αcomputer␈α
systems␈αthat␈αwork␈α
with␈αnatural␈α
language␈αand␈α
who␈αwrite␈αpapers␈α
on
␈↓ α∧␈↓the␈αsemantics␈αof␈αnatural␈αlanguage.␈α The␈αactual␈αyoung␈αcomputer␈αprogrammers␈αwho␈αcall␈αthemselves
␈↓ α∧␈↓hackers␈αand␈αwho␈αcome␈αclosest␈αto␈αmeeting␈αWeizenbaum's␈αdescription␈αdon't␈αwrite␈αpapers␈αon␈αnatural
␈↓ α∧␈↓language.␈α
 So␈αit␈α
seems␈αthat␈α
the␈αhackers␈α
whose␈α
work␈αneed␈α
not␈αbe␈α
studied␈αare␈α
Winograd,␈αSchank,␈α
et.


␈↓ α∧␈↓␈↓ ε|7␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓al.␈α who␈αare␈αprofessors␈αand␈αsenior␈αscientists.␈α The␈αChomsky␈αschool␈αmay␈αbe␈αembarassed␈αby␈αthe␈α
fact
␈↓ α∧␈↓that␈αit␈αhas␈αonly␈αrecently␈αarrived␈αat␈αthe␈αconclusion␈αthat␈αthe␈αsemantics␈αof␈αnatural␈αlanguage␈α
is␈αmore
␈↓ α∧␈↓fundamental␈αthan␈αits␈αsyntax,␈αwhile␈αAI␈αbased␈αresearchers␈αhave␈αbeen␈αpursuing␈αthis␈αline␈αfor␈αfifteen
␈↓ α∧␈↓years.

␈↓ α∧␈↓␈↓ αTThe␈α∂outside␈α⊂observer␈α∂should␈α⊂be␈α∂aware␈α⊂that␈α∂to␈α⊂some␈α∂extent␈α⊂this␈α∂is␈α⊂a␈α∂pillow␈α⊂fight␈α∂within
␈↓ α∧␈↓M.I.T.␈α Chomsky␈αand␈αHalle␈αare␈αnot␈αto␈αbe␈αdislodged␈αfrom␈αM.I.T.␈α and␈αneither␈αis␈αMinsky␈α
-␈αwhose
␈↓ α∧␈↓students␈α∞have␈α∞pioneered␈α∞the␈α∞AI␈α∞approach␈α∞to␈α∞natural␈α∞language.␈α∞ Schank␈α∞is␈α∞quite␈α∞secure␈α∂at␈α∞Yale.
␈↓ α∧␈↓Weizenbaum␈αalso␈α
has␈αtenure.␈α
 However,␈αsome␈αassistant␈α
professorships␈αmay␈α
be␈αat␈α
stake,␈αespecially
␈↓ α∧␈↓at M.I.T.

␈↓ α∧␈↓␈↓ αTAllen␈αNewell␈αand␈αHerbert␈αSimon␈αare␈αcriticized␈αfor␈αbeing␈αoveroptimistic␈αand␈αare␈αconsidered
␈↓ α∧␈↓morally␈α∂defective␈α∂for␈α∂attempting␈α∂to␈α∂describe␈α∂humans␈α∂as␈α∂difference-reducing␈α⊂machines.␈α∂ Simon's
␈↓ α∧␈↓view␈α
that␈α
the␈α
human␈α
is␈α
a␈α
simple␈α
system␈αin␈α
a␈α
complex␈α
environment␈α
is␈α
singled␈α
out␈α
for␈α
attack.␈α In␈α
my
␈↓ α∧␈↓opinion,␈α
they␈α
were␈α
overoptimistic,␈α
because␈α
their␈α
GPS␈α
model␈α
on␈α
which␈α
they␈α
put␈α
their␈α
bets␈αwasn't
␈↓ α∧␈↓good␈α⊃enough.␈α∩ Maybe␈α⊃Newell's␈α⊃current␈α∩␈↓↓production␈α⊃system␈α∩models␈↓␈α⊃will␈α⊃work␈α∩out␈α⊃better.␈α∩ As␈α⊃to
␈↓ α∧␈↓whether human mental structure will eventually turn out to be simple, I vacillate.

␈↓ α∧␈↓␈↓ αTI␈αregard␈αForrester's␈αmodels␈αas␈αincapable␈αof␈αtaking␈αinto␈αaccount␈αqualitative␈αchanges,␈αand␈αthe
␈↓ α∧␈↓world␈α∩models␈α∩they␈α∩have␈α∩built␈α∩as␈α∩defective␈α⊃even␈α∩in␈α∩their␈α∩own␈α∩terms,␈α∩because␈α∩they␈α∩leave␈α⊃out
␈↓ α∧␈↓saturation-of-demand␈α
effects␈α
that␈α
cannot␈α
be␈α
discovered␈α
by␈α
curve-fitting␈α
as␈α
long␈α
as␈α
a␈α
sytem␈αis␈α
rate-
␈↓ α∧␈↓of-expansion␈α
limited.␈α∞ Moreover,␈α
I␈α∞don't␈α
accept␈α∞his␈α
claim␈α
that␈α∞his␈α
models␈α∞are␈α
better␈α∞suited␈α
than
␈↓ α∧␈↓the␈α
unaided␈α
mind␈α
in␈α
"interpreting␈α
how␈α
social␈α
systems␈α
behave",␈α
but␈α
Weizenbaum's␈α
sarcasm␈αon␈α
page
␈↓ α∧␈↓246␈α
is␈α
unconvincing.␈α
 He␈α
quotes␈α
Forrester,␈α
"[desirable␈α
modes␈α
of␈α
behavior␈α
of␈α
the␈α
social␈α
system]␈α
␈↓↓seem
␈↓ α∧␈↓↓to␈α⊂be␈α∂possible␈α⊂only␈α∂if␈α⊂we␈α∂have␈α⊂a␈α∂good␈α⊂understanding␈α∂of␈α⊂the␈α∂system␈α⊂dynamics␈α∂and␈α⊂are␈α⊂willing␈α∂to
␈↓ α∧␈↓↓endure␈α∞the␈α∞self-discipline␈α∞and␈α∞pressures␈α∂that␈α∞must␈α∞accompany␈α∞the␈α∞desirable␈α∂mode"␈↓.␈α∞ Weizenbaum
␈↓ α∧␈↓comments,␈α∂␈↓↓"There␈α∂is␈α∂undoubtedly␈α∂some␈α∂interpretation␈α∂of␈α∂the␈α∂words␈α∂'system'␈α∂and␈α⊂'dynamics'␈α∂which
␈↓ α∧␈↓↓would␈α∞lend␈α∞a␈α∞benign␈α∞meaning␈α∞to␈α∞this␈α∞observation"␈↓.␈α
 Sorry,␈α∞but␈α∞it␈α∞looks␈α∞ok␈α∞to␈α∞me␈α∞provided␈α∞one␈α
is
␈↓ α∧␈↓suitably␈α∞critical␈α∂of␈α∞Forrester's␈α∞proposed␈α∂social␈α∞goals␈α∂and␈α∞the␈α∞possibility␈α∂of␈α∞making␈α∂the␈α∞necessary
␈↓ α∧␈↓assumptions and putting them into his models.

␈↓ α∧␈↓␈↓ αTSkinner's␈αbehaviorism␈αthat␈αrefuses␈αto␈αassign␈αreality␈αto␈αpeople's␈αinternal␈αstate␈αseems␈αwrong␈αto
␈↓ α∧␈↓me, but we can't call him immoral for trying to convince us of what he thinks is true.

␈↓ α∧␈↓␈↓ αTWeizenbaum␈α
quotes␈α
Edward␈α
Fredkin,␈α
former␈α
director␈α
of␈α
Project␈α
MAC,␈α
and␈α
the␈α
late␈α
Warren
␈↓ α∧␈↓McCulloch␈α⊂of␈α∂M.I.T.␈α⊂without␈α∂giving␈α⊂their␈α∂names.␈α⊂pp.␈α∂241␈α⊂and␈α∂240.␈α⊂ Perhaps␈α∂he␈α⊂thinks␈α⊂a␈α∂few
␈↓ α∧␈↓puzzles␈α∩will␈α∩make␈α∩the␈α∩book␈α⊃more␈α∩interesting,␈α∩and␈α∩this␈α∩is␈α⊃so.␈α∩ Fredkin's␈α∩plea␈α∩for␈α∩research␈α⊃in
␈↓ α∧␈↓automatic␈α
programming␈α
seems␈αto␈α
overestimate␈α
the␈αextent␈α
to␈α
which␈αour␈α
society␈α
currently␈α
relies␈αon
␈↓ α∧␈↓computers␈α
for␈α
decisions.␈α
 It␈α
also␈α
overestimates␈α
the␈α
ability␈α
of␈α
the␈α
faculty␈α
of␈α
a␈α
particular␈α
university␈α
to
␈↓ α∧␈↓control␈α
the␈α∞uses␈α
to␈α∞which␈α
technology␈α
will␈α∞be␈α
put,␈α∞and␈α
it␈α
underestimates␈α∞the␈α
difficulty␈α∞of␈α
making
␈↓ α∧␈↓knowledge␈α∞based␈α∞systems␈α∞of␈α∂practical␈α∞use.␈α∞ Weizenbaum␈α∞is␈α∂correct␈α∞in␈α∞pointing␈α∞out␈α∂that␈α∞Fredkin
␈↓ α∧␈↓doesn't␈α∞mention␈α∞the␈α∞existence␈α∂of␈α∞genuine␈α∞conflicts␈α∞in␈α∞society,␈α∂but␈α∞only␈α∞the␈α∞new␈α∂left␈α∞sloganeering
␈↓ α∧␈↓elsewhere␈αin␈αthe␈αbook␈αgives␈αa␈αhint␈αas␈αto␈αwhat␈αhe␈αthinks␈αthey␈αare␈αand␈αhow␈αhe␈αproposes␈αto␈αresolve
␈↓ α∧␈↓them.

␈↓ α∧␈↓␈↓ αTThe␈α⊂quote␈α⊂from␈α⊂McCulloch␈α⊂is␈α⊂from␈α∂an␈α⊂essay␈α⊂entitled␈α⊂"Toward␈α⊂some␈α⊂circuitry␈α⊂of␈α∂ethical
␈↓ α∧␈↓robots..."(very␈α
long␈α
title)␈α
ACTA␈α
BIOTHEORETICA␈α
XI␈α
147-156,␈α
1956.␈α
(Minsky␈α
tells␈α
me␈α
"this␈αis␈α
a


␈↓ α∧␈↓␈↓ ε|8␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓brave␈αattempt␈αto␈αfind␈α
a␈αdignified␈αsense␈αof␈α
freedom␈αwithin␈αthe␈αpsychological␈αdeterminism␈α
morass.")
␈↓ α∧␈↓I␈αthink␈αthis␈αcan␈αbe␈αdone␈αbetter␈αnow,␈αbut␈αWeizenbaum␈αimplies␈αthat␈αhis␈α1956␈αeffort␈αis␈αto␈αhis␈αmoral
␈↓ α∧␈↓discredit.

␈↓ α∧␈↓␈↓ αTFinally,␈α
Weizenbaum␈α
attributes␈α
to␈αme␈α
two␈α
statements␈α
-␈αboth␈α
from␈α
oral␈α
presentations␈α-␈α
which
␈↓ α∧␈↓I␈α
cannot␈α∞verify.␈α
 One␈α
of␈α∞them␈α
is␈α
␈↓↓"The␈α∞only␈α
reason␈α
we␈α∞have␈α
not␈α
yet␈α∞succeeded␈α
in␈α∞simulating␈α
every
␈↓ α∧␈↓↓aspect␈αof␈αthe␈αreal␈αworld␈αis␈αthat␈αwe␈αhave␈αbeen␈αlacking␈αa␈αsufficiently␈αpowerful␈αlogical␈αcalculus.␈α I␈αam
␈↓ α∧␈↓↓working␈α∞on␈α∂that␈α∞problem."␈↓␈α∞This␈α∂statement␈α∞doesn't␈α∞express␈α∂my␈α∞present␈α∞opinion␈α∂or␈α∞my␈α∂opinion␈α∞in
␈↓ α∧␈↓1973␈αwhen␈αI␈αam␈αalleged␈αto␈αhave␈αexpressed␈αit␈αin␈αa␈αdebate,␈αand␈αno-one␈αhas␈αbeen␈αable␈αto␈αfind␈αit␈αin
␈↓ α∧␈↓the video-tape of the debate.

␈↓ α∧␈↓␈↓ αTWe␈α
can't␈α
simulate␈α∞"every␈α
aspect␈α
of␈α∞the␈α
real␈α
world",␈α
because␈α∞the␈α
initial␈α
state␈α∞information␈α
is
␈↓ α∧␈↓never␈α
available,␈αthe␈α
laws␈α
of␈αmotion␈α
are␈α
imperfectly␈αknown,␈α
and␈α
the␈αcalculations␈α
for␈α
a␈αsimulation
␈↓ α∧␈↓are␈α∞too␈α∞extensive.␈α∞ Moreover,␈α∞simulation␈α∞wouldn't␈α∞necessarily␈α∞answer␈α∞our␈α∞questions.␈α∞ Instead,␈α∞we
␈↓ α∧␈↓must␈α
find␈α
out␈α
how␈αto␈α
represent␈α
in␈α
the␈αmemory␈α
of␈α
a␈α
computer␈αthe␈α
information␈α
about␈α
the␈αreal␈α
world
␈↓ α∧␈↓that␈αis␈αactually␈αavailable␈αto␈αa␈αmachine␈αor␈α
organism␈αwith␈αgiven␈αsensory␈αcapability,␈αand␈αalso␈αhow␈α
to
␈↓ α∧␈↓represent␈αa␈α
means␈αof␈αdrawing␈α
those␈αuseful␈αconclusions␈α
about␈αthe␈αeffects␈α
of␈αcourses␈αof␈α
action␈αthat
␈↓ α∧␈↓can␈αbe␈α
correctly␈αinferred␈αfrom␈α
the␈αattainable␈αinformation.␈α
 Having␈αa␈α␈↓↓sufficiently␈α
powerful␈αlogical
␈↓ α∧␈↓↓calculus␈↓ is an important part of this problem - but one of the easier parts.

␈↓ α∧␈↓␈↓ αTThe␈α∞second␈α∂quotation␈α∞from␈α∂me␈α∞is␈α∂the␈α∞rhetorical␈α∞question,␈α∂␈↓↓"What␈α∞do␈α∂judges␈α∞know␈α∂that␈α∞we
␈↓ α∧␈↓↓cannot tell a computer"␈↓.  I'll stand on that if we make it "eventually tell".

␈↓ α∧␈↓α␈↓ ¬↓A SUMMARY OF POLEMICAL SINS

␈↓ α∧␈↓␈↓ αTThe␈α
speculative␈αsections␈α
of␈αthe␈α
book␈αcontain␈α
numerous␈αdubious␈α
little␈αtheories,␈α
such␈α
as␈αthis
␈↓ α∧␈↓one␈α
about␈αthe␈α
dehumanizing␈α
effect␈αof␈α
of␈αthe␈α
invention␈α
of␈αthe␈α
clock:␈α
␈↓↓"The␈αclock␈α
had␈αcreated␈α
literally
␈↓ α∧␈↓↓a␈α
new␈αreality;␈α
and␈α
that␈αis␈α
what␈α
I␈αmeant␈α
when␈αI␈α
said␈α
earlier␈αthat␈α
the␈α
trick␈αman␈α
turned␈αthat␈α
prepared
␈↓ α∧␈↓↓the␈αscene␈αfor␈α
the␈αrise␈αof␈α
modern␈αscience␈αwas␈αnothing␈α
less␈αthan␈αthe␈α
transformation␈αof␈αnature␈α
and␈αof
␈↓ α∧␈↓↓his␈αperception␈αof␈αreality.␈α It␈αis␈αimportant␈αto␈αrealize␈αthat␈αthis␈αnewly␈αcreated␈αreality␈αwas␈αand␈αremains
␈↓ α∧␈↓↓an␈α
impoverished␈α
version␈α
of␈αthe␈α
older␈α
one,␈α
for␈αit␈α
rests␈α
on␈α
a␈αrejection␈α
of␈α
those␈α
direct␈α
experiences␈αthat
␈↓ α∧␈↓↓formed␈αthe␈αbasis␈αfor,␈αand␈αindeed␈αconstituted␈αthe␈α
old␈αreality.␈α The␈αfeeling␈αof␈αhunger␈αwas␈αrejected␈αas␈α
a
␈↓ α∧␈↓↓stimulus␈αfor␈α
eating;␈αinstead␈αone␈α
ate␈αwhen␈α
an␈αabstract␈αmodel␈α
had␈αachieved␈αa␈α
certain␈αstate,␈α
i.e.␈αwhen
␈↓ α∧␈↓↓the␈α
hand␈α
of␈α
a␈α
clock␈α
pointed␈α
to␈α
certain␈α
marks␈α
on␈α
the␈α
clock's␈α
face␈α
(the␈α
anthropomorphism␈α
here␈α
is␈α
highly
␈↓ α∧␈↓↓significant too), and similarly for signals for sleep and rising, and so on."␈↓

␈↓ α∧␈↓␈↓ αTThis␈αidealization␈α
of␈αprimitive␈αlife␈α
is␈αsimply␈αthoughtless.␈α
 Like␈αmodern␈αman,␈α
primitive␈αman
␈↓ α∧␈↓ate␈αwhen␈αthe␈αfood␈αwas␈αready,␈αand␈αprimitive␈αman␈αprobably␈αhad␈αto␈αstart␈αpreparing␈αit␈αeven␈αfurther
␈↓ α∧␈↓in␈α
advance.␈α
 Like␈α
modern␈αman,␈α
primitive␈α
man␈α
lived␈α
in␈αfamilies␈α
whose␈α
members␈α
are␈α
no␈αmore␈α
likely
␈↓ α∧␈↓to become hungry all at once than are the members of a present family.

␈↓ α∧␈↓␈↓ αTI␈αget␈αthe␈αfeeling␈αthat␈αin␈αtoppling␈αthis␈αmicrotheory␈αI␈αam␈αnot␈αplaying␈αthe␈αgame;␈αthe␈αtheory␈αis
␈↓ α∧␈↓intended␈α
only␈αto␈α
provide␈α
an␈αatmosphere,␈α
and␈α
like␈αthe␈α
reader␈α
of␈αa␈α
novel,␈α
I␈αam␈α
supposed␈αto␈α
suspend
␈↓ α∧␈↓disbelief.␈α∂ But␈α∞the␈α∂contention␈α∞that␈α∂science␈α∞has␈α∂driven␈α∞us␈α∂from␈α∞a␈α∂psychological␈α∞Garden␈α∂of␈α∞Eden
␈↓ α∧␈↓depends heavily on such word pictures.

␈↓ α∧␈↓␈↓ αTBy␈α∞the␈α∞way,␈α∞I␈α∞recall␈α∞from␈α∞my␈α∞last␈α∞sabbatical␈α∞at␈α∞M.I.T.␈α∞that␈α∞the␈α∞␈↓↓feeling␈α∞of␈α∞hunger␈↓␈α∞is␈α
more


␈↓ α∧␈↓␈↓ ε|9␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓often␈α
the␈α
␈↓↓direct␈α∞social␈α
stimulus␈α
for␈α
eating␈↓␈α∞for␈α
the␈α
"hackers"␈α
deplored␈α∞in␈α
Chapter␈α
4␈α
than␈α∞it␈α
could
␈↓ α∧␈↓have␈α∂been␈α∞for␈α∂primitive␈α∞man.␈α∂ Often␈α∞on␈α∂a␈α∂crisp␈α∞New␈α∂England␈α∞night,␈α∂even␈α∞as␈α∂the␈α∂clock␈α∞strikes
␈↓ α∧␈↓three,␈αI␈αhear␈αthem␈αcall␈αto␈αone␈αanother,␈αmessages␈αflash␈αon␈αthe␈αscreens,␈αa␈αflock␈αof␈αhackers␈αmagically
␈↓ α∧␈↓gathers, and the whole picturesque assembly rushes chattering off to Chinatown.

␈↓ α∧␈↓␈↓ αTI find the book substandard as a piece of polemical writing in the following respects:

␈↓ α∧␈↓␈↓ αT1.␈αThe␈αauthor␈αhas␈αfailed␈αto␈αwork␈αout␈αhis␈αown␈αpositions␈αon␈αthe␈αissues␈αhe␈αdiscusses.␈α Making
␈↓ α∧␈↓an␈α
extreme␈α∞statement␈α
in␈α∞one␈α
place␈α∞and␈α
a␈α∞contradictory␈α
statement␈α∞in␈α
another␈α∞is␈α
no␈α∞substitute␈α
for
␈↓ α∧␈↓trying␈αto␈α
take␈αall␈αthe␈α
factors␈αinto␈αaccount␈α
and␈αreach␈αa␈α
considered␈αposition.␈α
 Unsuspicious␈αreaders
␈↓ α∧␈↓can␈αcome␈αaway␈αwith␈αa␈αgreat␈αvariety␈αof␈αviews,␈αand␈αthe␈αbook␈αcan␈αbe␈αused␈αto␈α
support␈αcontradictory
␈↓ α∧␈↓positions.

␈↓ α∧␈↓␈↓ αT2.␈α⊂The␈α∂computer␈α⊂linguists␈α∂-␈α⊂Winograd,␈α⊂Schank,␈α∂et.␈α⊂al.␈α∂ -␈α⊂are␈α∂denigrated␈α⊂as␈α⊂hackers␈α∂and
␈↓ α∧␈↓compulsive computer programmers by innuendo.

␈↓ α∧␈↓␈↓ αT3.␈α∂One␈α∞would␈α∂like␈α∞to␈α∂know␈α∞more␈α∂precisely␈α∞what␈α∂biological␈α∞and␈α∂psychological␈α∞experiments
␈↓ α∧␈↓and␈α∩computer␈α∩applications␈α∩he␈α∩finds␈α∩acceptable.␈α∩ Reviewers␈α∩have␈α∩already␈α∩drawn␈α∩a␈α∩variety␈α⊃of
␈↓ α∧␈↓conclusions on this point.

␈↓ α∧␈↓␈↓ αT4.␈α
The␈α
terms␈α
"authentic",␈α"obscene",␈α
and␈α
"dehumanization␈α
are␈αused␈α
as␈α
clubs.␈α
 This␈α
is␈αwhat
␈↓ α∧␈↓mathematicians call "proof by intimidation".

␈↓ α∧␈↓␈↓ αT5.␈α∞The␈α∂book␈α∞encourages␈α∂a␈α∞snobbery␈α∞that␈α∂has␈α∞no␈α∂need␈α∞to␈α∞argue␈α∂for␈α∞its␈α∂point␈α∞of␈α∂view␈α∞but
␈↓ α∧␈↓merely␈αutters␈αcode␈αwords,␈αon␈αhearing␈αwhich␈αthe␈αaudience␈αis␈αsupposed␈αapplaud␈αor␈αhiss␈αas␈αthe␈αcase
␈↓ α∧␈↓may be.  The ␈↓↓New Scientist␈↓ reviewer certainly salivates in most of the intended places.

␈↓ α∧␈↓␈↓ αT6.␈α⊂Finally,␈α⊂when␈α⊃moralizing␈α⊂is␈α⊂both␈α⊂vehement␈α⊃and␈α⊂vague,␈α⊂it␈α⊂invites␈α⊃authoritarian␈α⊂abuse
␈↓ α∧␈↓either␈α
by␈α∞existing␈α
authority␈α
or␈α∞by␈α
new␈α
political␈α∞movements.␈α
 Imagine,␈α
if␈α∞you␈α
can,␈α
that␈α∞this␈α
book
␈↓ α∧␈↓were␈α⊂the␈α⊂bible␈α⊂of␈α∂some␈α⊂bureaucracy,␈α⊂e.g.␈α⊂ the␈α∂Office␈α⊂of␈α⊂Technology␈α⊂Assessment,␈α⊂that␈α∂acquired
␈↓ α∧␈↓power␈α∩over␈α∩the␈α∩computing␈α∩or␈α∩scientific␈α∩activities␈α∩of␈α∩a␈α∩university,␈α∩state,␈α∩or␈α∩country.␈α∩ Suppose
␈↓ α∧␈↓Weizenbaum's␈αslogans␈αwere␈αcombined␈αwith␈α␈↓↓the␈αbureaucratic␈αethic␈↓␈αthat␈αholds␈αthat␈αany␈αproblem␈αcan
␈↓ α∧␈↓be␈αsolved␈αby␈αa␈αlaw␈αforbidding␈αsomething␈αand␈αa␈αbureaucracy␈αof␈αeager␈αyoung␈αlawyers␈αto␈αenforce␈αit.
␈↓ α∧␈↓Postulate␈α
further␈α∞a␈α
vague␈α
␈↓↓Humane␈α∞Research␈α
Act␈↓␈α∞and␈α
a␈α
"public␈α∞interest"␈α
organization␈α∞with␈α
more
␈↓ α∧␈↓eager␈αyoung␈αlawyers␈αsuing␈αto␈α
get␈αjudges␈αto␈αlegislate␈αnew␈αinterpretations␈α
of␈αthe␈αAct.␈α One␈αcan␈αsee␈α
a
␈↓ α∧␈↓laboratory␈α
needing␈α
more␈α
lawyers␈α
than␈αscientists␈α
and␈α
a␈α
Humane␈α
Research␈α
Administrator␈αcapable␈α
of
␈↓ α∧␈↓forbidding or requiring almost anything.

␈↓ α∧␈↓␈↓ αTI␈αsee␈αno␈αevidence␈αthat␈αWeizenbaum␈αforsees␈αhis␈αwork␈αbeing␈αused␈αin␈αthis␈αway;␈αhe␈αdoesn't␈αuse
␈↓ α∧␈↓the␈αphrase␈α␈↓↓laissez␈αinnover␈↓␈αwhich␈αis␈αthe␈αwould-be␈αscience␈αbureaucrat's␈αanalogue␈αof␈αthe␈αeconomist's
␈↓ α∧␈↓␈↓↓laissez␈α
faire␈↓,␈α
and␈α
he␈αnever␈α
uses␈α
the␈α
indefinite␈αphrase␈α
"it␈α
should␈α
be␈αdecided"␈α
which␈α
is␈α
a␈αcommon
␈↓ α∧␈↓expression␈α∪of␈α∀the␈α∪bureaucratic␈α∪ethic.␈α∀ However,␈α∪he␈α∪has␈α∀certainly␈α∪given␈α∪his␈α∀fellow␈α∪computer
␈↓ α∧␈↓scientists at least some reason to worry about potential tyranny.

␈↓ α∧␈↓␈↓ αTLet␈αme␈αconclude␈αthis␈αsection␈αwith␈αa␈αquotation␈αfrom␈αAndrew␈αD.␈αWhite,␈αthe␈αfirst␈αpresident␈αof
␈↓ α∧␈↓Cornell␈α
University,␈αthat␈α
seems␈αapplicable␈α
to␈α
the␈αpresent␈α
situation␈α-␈α
not␈αonly␈α
in␈α
computer␈αscience,
␈↓ α∧␈↓but␈α
also␈α
in␈α
biology.␈α
-␈α
"␈↓↓In␈α
all␈α
modern␈α
history,␈α
interference␈α
with␈α
science␈α
in␈α
the␈α
supposed␈α∞interest␈α
of


␈↓ α∧␈↓␈↓ εu10␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓↓religion,␈α
no␈αmatter␈α
how␈α
conscientious␈αsuch␈α
interference␈α
may␈αhave␈α
been,␈α
has␈αresulted␈α
in␈α
the␈αdirest␈α
evils
␈↓ α∧␈↓↓both␈α
to␈α
religion␈α∞and␈α
to␈α
science,␈α
and␈α∞invariably;␈α
and,␈α
on␈α
the␈α∞other␈α
hand,␈α
all␈α∞untrammelled␈α
scientific
␈↓ α∧␈↓↓investigation,␈αno␈αmatter␈αhow␈αdangerous␈αto␈αreligion␈αsome␈αof␈αits␈αstages␈αmy␈αhave␈αseemed␈αfor␈αthe␈αtime␈αto
␈↓ α∧␈↓↓be,␈αhas␈α
invariably␈αresulted␈α
in␈αthe␈α
highest␈αgood␈α
both␈αof␈α
religion␈αand␈α
of␈αscience"␈↓.␈α
 Substitute␈α␈↓↓morality␈↓
␈↓ α∧␈↓for␈α␈↓↓religion␈↓␈α
and␈αthe␈α
parallel␈αis␈α
clear.␈α Frankly,␈αthe␈α
feebleness␈αof␈α
the␈αreaction␈α
to␈αattacks␈αon␈α
scientific
␈↓ α∧␈↓freedom worries me more than the strength of the attacks.

␈↓ α∧␈↓α␈↓ βEWHAT WORRIES  ABOUT COMPUTERS ARE WARRANTED?

␈↓ α∧␈↓␈↓ αT Grumbling␈αabout␈αWeizenbaum's␈αmistakes␈αand␈αmoralizing␈αis␈αnot␈αenough.␈α Genuine␈αworries
␈↓ α∧␈↓prompted␈α⊂the␈α⊂book,␈α⊂and␈α⊂many␈α∂people␈α⊂share␈α⊂them.␈α⊂ Here␈α⊂are␈α∂the␈α⊂genuine␈α⊂concerns␈α⊂that␈α⊂I␈α∂can
␈↓ α∧␈↓identify␈α
and␈α
the␈α
opinions␈αof␈α
one␈α
computer␈α
scientist␈α
about␈αtheir␈α
resolution:␈α
What␈α
is␈α
the␈αdanger␈α
that
␈↓ α∧␈↓the␈α⊂computer␈α∂will␈α⊂lead␈α⊂to␈α∂a␈α⊂false␈α⊂model␈α∂of␈α⊂man?␈α⊂ What␈α∂is␈α⊂the␈α⊂danger␈α∂that␈α⊂computers␈α⊂will␈α∂be
␈↓ α∧␈↓misused?␈α∃ Can␈α∃human-level␈α∃artificial␈α∃intelligence␈α∃be␈α∃achieved?␈α∃ What,␈α∃if␈α∃any,␈α∀motivational
␈↓ α∧␈↓characteristics␈α
will␈α
it␈α
have?␈α
 Would␈α
the␈α
achievement␈α
of␈α
artificial␈α
intelligence␈α
be␈α
good␈α
or␈α
bad␈αfor
␈↓ α∧␈↓humanity?


␈↓ α∧␈↓α␈↓ αT1. Does the computer model lead to a false model of man.

␈↓ α∧␈↓␈↓ αTHistorically,␈α
the␈α∞mechanistic␈α
model␈α∞of␈α
the␈α∞life␈α
and␈α∞the␈α
world␈α∞followed␈α
animistic␈α∞models␈α
in
␈↓ α∧␈↓accordance␈αwith␈α
which,␈αpriests␈α
and␈αmedicine␈α
men␈αtried␈α
to␈αcorrect␈α
malfunctions␈αof␈αthe␈α
environment
␈↓ α∧␈↓and␈α∞man␈α∞by␈α∞inducing␈α∞spirits␈α∞to␈α
behave␈α∞better.␈α∞ Replacing␈α∞them␈α∞by␈α∞mechanistic␈α∞models␈α
replaced
␈↓ α∧␈↓shamanism␈α⊂by␈α∂medicine.␈α⊂ Roszak␈α∂explicitly␈α⊂would␈α∂like␈α⊂to␈α∂bring␈α⊂these␈α∂models␈α⊂back,␈α⊂because␈α∂he
␈↓ α∧␈↓finds␈α
them␈α
more␈α
"human",␈α
but␈α
he␈α
ignores␈α
the␈α
sad␈α
fact␈α
that␈α
they␈α
don't␈α
work,␈α
because␈α
the␈α
world␈α
isn't
␈↓ α∧␈↓constructed␈α∞that␈α∞way.␈α∂ The␈α∞pre-computer␈α∞mechanistic␈α∂models␈α∞of␈α∞the␈α∂mind␈α∞were,␈α∞in␈α∂my␈α∞opinion,
␈↓ α∧␈↓unsuccessful,but␈αI␈αthink␈αthe␈αpsychologists␈αpursuing␈αcomputational␈αmodels␈αof␈αmental␈αprocesses␈αmay
␈↓ α∧␈↓eventually develop a really beneficial psychiatry.

␈↓ α∧␈↓␈↓ αTPhilosophical␈α⊂and␈α∂moral␈α⊂thinking␈α∂has␈α⊂never␈α∂found␈α⊂a␈α∂model␈α⊂of␈α∂man␈α⊂that␈α⊂relates␈α∂human
␈↓ α∧␈↓beliefs␈αand␈αpurposes␈αto␈αthe␈αphysical␈αworld␈αin␈αa␈αplausible␈αway.␈α Some␈αof␈αthe␈αunsuccessful␈αattempts
␈↓ α∧␈↓have␈αbeen␈αmore␈α
mechanistic␈αthan␈αothers.␈α
 Both␈αmechanistic␈αand␈α
non-mechanistic␈αmodels␈αhave␈α
led
␈↓ α∧␈↓to␈α⊂great␈α⊂harm␈α⊂when␈α⊂made␈α⊂the␈α⊂basis␈α⊂of␈α⊂political␈α⊂ideology,␈α⊂because␈α⊂they␈α⊂have␈α⊃allowed␈α⊂tortuous
␈↓ α∧␈↓reasoning␈αto␈αjustify␈α
actions␈αthat␈αsimple␈α
human␈αintuition␈αregards␈αas␈α
immoral.␈α In␈αmy␈α
opinion,␈αthe
␈↓ α∧␈↓relation␈αbetween␈αbeliefs,␈αpurposes␈αand␈αwants␈αto␈αthe␈αphysical␈αworld␈αis␈αa␈αcomplicated␈αbut␈αultimately
␈↓ α∧␈↓solvable␈α
problem.␈α Computer␈α
models␈αcan␈α
help␈αsolve␈α
it,␈αand␈α
can␈αprovide␈α
criteria␈αthat␈α
will␈αenable␈α
us
␈↓ α∧␈↓to␈αreject␈αfalse␈αsolutions.␈α The␈αlatter␈αis␈α
more␈αimportant␈αfor␈αnow,␈αand␈αcomputer␈αmodels␈α
are␈αalready
␈↓ α∧␈↓hastening the decay of dialectical materialism in the Soviet Union.


␈↓ α∧␈↓α␈↓ αT2. What is the danger that computers will be misused?

␈↓ α∧␈↓␈↓ αTUp␈αto␈αnow,␈αcomputers␈α
have␈αbeen␈αjust␈αanother␈α
labor-saving␈αtechnology.␈α I␈αdon't␈α
agree␈αwith
␈↓ α∧␈↓Weizenbaum's␈α
acceptance␈αof␈α
the␈αclaim␈α
that␈αour␈α
society␈α
would␈αhave␈α
been␈αinundated␈α
by␈αpaper␈α
work
␈↓ α∧␈↓without␈αcomputers.␈α Without␈αcomputers,␈αpeople␈αwould␈αwork␈αa␈αlittle␈αharder␈αand␈αget␈αa␈αlittle␈αless␈αfor
␈↓ α∧␈↓their␈αwork.␈α However,␈αwhen␈αhome␈αterminals␈αbecome␈αavailable,␈αsocial␈αchanges␈αof␈αthe␈αmagnitude␈αof
␈↓ α∧␈↓those␈α
produced␈αby␈α
the␈αtelephone␈α
and␈αautomobile␈α
will␈α
occur.␈α I␈α
have␈αdiscussed␈α
them␈αelsewhere,␈α
and


␈↓ α∧␈↓␈↓ εu11␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓I␈α
think␈α
they␈α
will␈αbe␈α
good␈α
-␈α
as␈αwere␈α
the␈α
changes␈α
produced␈αby␈α
the␈α
automobile␈α
and␈α
the␈αtelephone.
␈↓ α∧␈↓Tyranny␈αcomes␈α
from␈αcontrol␈α
of␈αthe␈α
police␈αcoupled␈αwith␈α
a␈αtyrannical␈α
ideology;␈αdata␈α
banks␈αwill␈αbe␈α
a
␈↓ α∧␈↓minor convenience.  No dictatorship yet has been overthrown for lack of a data bank.

␈↓ α∧␈↓␈↓ αTOne's␈α∂estimate␈α∂of␈α∂whether␈α∂technology␈α∂will␈α∂work␈α∂out␈α∂well␈α∂in␈α∂the␈α∂future␈α∂is␈α∂correlated␈α∞with
␈↓ α∧␈↓one's␈αview␈αof␈αhow␈αit␈αworked␈αout␈αin␈αthe␈αpast.␈α I␈αthink␈αit␈αhas␈αworked␈αout␈αwell␈α-␈αe.g.␈αcars␈αwere␈αnot␈αa
␈↓ α∧␈↓mistake␈α-␈αand␈αam␈αoptimistic␈αabout␈αthe␈αfuture.␈α I␈αfeel␈αthat␈αmuch␈αcurrent␈αideology␈αis␈αa␈αcombination
␈↓ α∧␈↓of␈α∪older␈α∀anti-scientific␈α∪and␈α∪anti-technological␈α∀views␈α∪with␈α∪new␈α∀developments␈α∪in␈α∀the␈α∪political
␈↓ α∧␈↓technology of instigating and manipulating fears and guilt feelings.


␈↓ α∧␈↓α␈↓ αT3. What motivations will artificial intelligence have?

␈↓ α∧␈↓␈↓ αTIt␈αwill␈αhave␈αwhat␈αmotivations␈αwe␈αchoose␈αto␈αgive␈αit.␈α Those␈αwho␈αfinally␈αcreate␈αit␈αshould␈αstart
␈↓ α∧␈↓by␈αmotivating␈α
it␈αonly␈α
to␈αanswer␈α
questions␈αand␈α
should␈αhave␈α
the␈αsense␈α
to␈αask␈α
for␈αfull␈α
pictures␈αof␈α
the
␈↓ α∧␈↓consequences␈α
of␈α
alternate␈α
actions␈α
rather␈α
than␈α
simply␈α
how␈α
to␈α
achieve␈α
a␈α
fixed␈α
goal,␈α
ignoring␈α
possible
␈↓ α∧␈↓side-effects.␈α
 Giving␈α∞it␈α
human␈α
motivational␈α∞structure␈α
with␈α
its␈α∞shifting␈α
goals␈α
sensitive␈α∞to␈α
physical
␈↓ α∧␈↓state would require a deliberate effort beyond that required to make it behave intelligently.


␈↓ α∧␈↓α␈↓ αT4. Will artificial intelligence be good or bad?

␈↓ α∧␈↓␈↓ αTHere␈α⊂we␈α⊂are␈α∂talking␈α⊂about␈α⊂machines␈α⊂with␈α∂the␈α⊂same␈α⊂range␈α∂of␈α⊂intellectual␈α⊂abilities␈α⊂as␈α∂are
␈↓ α∧␈↓posessed␈α
by␈α
humans.␈α
 However,␈α
the␈α
science␈α
fiction␈α
vision␈α
of␈α
robots␈α
with␈α
almost␈α
precisely␈α
the␈α
ability
␈↓ α∧␈↓of␈α∀a␈α∀human␈α∀is␈α∀quite␈α∀unlikely,␈α∀because␈α∪the␈α∀next␈α∀generation␈α∀of␈α∀computers␈α∀or␈α∀even␈α∪hooking
␈↓ α∧␈↓computers␈α∩together␈α∩would␈α∩produce␈α∩an␈α∩intelligence␈α∪that␈α∩might␈α∩be␈α∩qualitatively␈α∩like␈α∩that␈α∪of␈α∩a
␈↓ α∧␈↓human,␈αbut␈αthousands␈α
of␈αtimes␈αfaster.␈α What␈α
would␈αit␈αbe␈α
like␈αto␈αbe␈αable␈α
to␈αput␈αa␈α
hundred␈αyears
␈↓ α∧␈↓thought␈α
into␈α
every␈α
decision?␈α
 I␈α
think␈α
it␈α
is␈α
impossible␈α
to␈α
say␈α
whether␈α
qualitatively␈α∞better␈α
answers
␈↓ α∧␈↓would be obtained; we will have to try it and see.

␈↓ α∧␈↓␈↓ αTThe␈α⊃achievement␈α⊂of␈α⊃above-human-level␈α⊃artificial␈α⊂intelligence␈α⊃will␈α⊂open␈α⊃to␈α⊃humanity␈α⊂an
␈↓ α∧␈↓incredible␈α∞variety␈α∂of␈α∞options.␈α∂ We␈α∞cannot␈α∞now␈α∂fully␈α∞envisage␈α∂what␈α∞these␈α∞options␈α∂will␈α∞be,␈α∂but␈α∞it
␈↓ α∧␈↓seems␈α
apparent␈αthat␈α
one␈α
of␈αthe␈α
first␈αuses␈α
of␈α
high-level␈αartificial␈α
intelligence␈αwill␈α
be␈α
to␈αdetermine
␈↓ α∧␈↓the␈α∞consequences␈α∞of␈α∞alternate␈α∂policies␈α∞governing␈α∞its␈α∞use.␈α∞ I␈α∂think␈α∞the␈α∞most␈α∞likely␈Nvariant␈α∂is␈α∞that
␈↓ α∧␈↓man␈α⊗will␈α⊗use␈α⊗artificial␈α↔intelligence␈α⊗to␈α⊗transform␈α⊗himself,␈α↔but␈α⊗once␈α⊗its␈α⊗properties␈α↔and␈α⊗the
␈↓ α∧␈↓conequences␈αof␈αits␈α
use␈αare␈αknown,␈αwe␈α
may␈αdecide␈αnot␈αto␈α
use␈αit.␈α Science␈αwould␈α
then␈αbe␈αa␈αsport␈α
like
␈↓ α∧␈↓mountain␈αclimbing;␈α
the␈αpoint␈αwould␈α
be␈αto␈αdiscover␈α
the␈αfacts␈αabout␈α
the␈αworld␈αusing␈α
some␈αstylized
␈↓ α∧␈↓limited␈α
means.␈α I␈α
wouldn't␈αlike␈α
that,␈αbut␈α
once␈αman␈α
is␈αconfronted␈α
by␈αthe␈α
actuality␈αof␈α
full␈α
AI,␈αthey
␈↓ α∧␈↓may␈αfind␈αour␈αopinion␈αas␈α
relevant␈αto␈αthem␈αas␈αwe␈α
would␈αfind␈αthe␈αopinion␈αof␈α␈↓↓Pithecanthropus␈↓␈α
about
␈↓ α∧␈↓whether subsequent evolution took the right course.


␈↓ α∧␈↓α␈↓ αT5. What shouldn't computers be programmed to do.

␈↓ α∧␈↓␈↓ αTObviously␈α_one␈α_shouldn't␈α↔program␈α_computers␈α_to␈α_do␈α↔things␈α_that␈α_shouldn't␈α_be␈α↔done.
␈↓ α∧␈↓Moreover,␈α∞we␈α∞shouldn't␈α∞use␈α∞programs␈α∞to␈α∞mislead␈α∞ourselves␈α∞or␈α∞other␈α∞people.␈α∞ Apart␈α∞from␈α∞that,␈α∞I
␈↓ α∧␈↓find␈α
none␈α
of␈α
Weizenbaum's␈α
examples␈α∞convincing.␈α
 However,␈α
I␈α
doubt␈α
the␈α
advisability␈α∞of␈α
making


␈↓ α∧␈↓␈↓ εu12␈↓ ∧
␈↓ α∧␈↓␈↓ ε	STILL A DRAFT␈↓ ∧


␈↓ α∧␈↓robots␈αwith␈αhuman-like␈αmotivational␈αand␈α
emotional␈αstructures␈αthat␈αmight␈αhave␈αrights␈α
and␈αduties
␈↓ α∧␈↓independently␈α⊂of␈α⊂humans.␈α⊂ Moreover,␈α⊂I␈α⊂think␈α⊂it␈α∂might␈α⊂be␈α⊂dangerous␈α⊂to␈α⊂make␈α⊂a␈α⊂machine␈α∂that
␈↓ α∧␈↓evolved␈α
intelligence␈α
by␈α
responding␈α
to␈α
a␈α∞program␈α
of␈α
rewards␈α
and␈α
punishments␈α
unless␈α∞its␈α
trainers
␈↓ α∧␈↓understand the intellectual and motivational structure being evolved.

␈↓ α∧␈↓␈↓ αTAll␈αthese␈αquestions␈αmerit␈αand␈αhave␈αreceived␈αmore␈αextensive␈αdiscussion,␈αbut␈αI␈αthink␈αthe␈αonly
␈↓ α∧␈↓rational␈α∞policy␈α∞now␈α∞is␈α∞to␈α∞expect␈α∞the␈α∞people␈α∞confronted␈α∞by␈α∞the␈α∞problem␈α∞to␈α∞understand␈α∞their␈α
best
␈↓ α∧␈↓interests␈α
better␈α
than␈α
we␈αnow␈α
can.␈α
 Even␈α
if␈α
full␈αAI␈α
were␈α
to␈α
arrive␈αnext␈α
year,␈α
this␈α
would␈α
be␈αright.
␈↓ α∧␈↓Correct␈α
decisions␈αwill␈α
require␈αan␈α
intense␈αeffort␈α
that␈α
cannot␈αbe␈α
mobilized␈αto␈α
consider␈αan␈α
eventuality
␈↓ α∧␈↓that␈α∞is␈α∂still␈α∞remote.␈α∞ Imagine␈α∂asking␈α∞the␈α∞presidential␈α∂candidates␈α∞to␈α∞debate␈α∂on␈α∞TV␈α∞what␈α∂each␈α∞of
␈↓ α∧␈↓them would do about each of the forms that full AI might take.

␈↓ α∧␈↓␈↓ αT␈↓εThis␈α
review␈αis␈α
filed␈αas␈α
WEIZEN.REV[PUB,JMC]␈α
at␈αSU-AI␈α
on␈αthe␈α
ARPA␈α
net.␈α Any␈α
comments␈αsent␈α
to
␈↓ α∧␈↓εJMC@SU-AI␈α
will␈α∞be␈α
stored␈α∞in␈α
the␈α∞directory␈α
PUB,JMC␈α∞also␈α
known␈α∞as␈α
␈↓↓McCarthy's␈α∞Electric␈α
Magazine␈↓ε.
␈↓ α∧␈↓εThe comment files will be designated WEIZEN.1, WEIZEN.2, etc.␈↓

␈↓ α∧␈↓John McCarthy
␈↓ α∧␈↓Artificial Intelligence Laboratory
␈↓ α∧␈↓Stanford, California 94305

␈↓ α∧␈↓March 25, 1976




























␈↓ α∧␈↓␈↓ εu13␈↓ ∧